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Preface 



These curriculum-based vocational assessment (CBV ,M training 
materials were developed via a three-year project funded by e Office of 
Special Education and Rehabilitative Services, United States Jepartment 
of Education (#G008630057). The underlying rationale for producing this 
CB VA training system arose primarily from the comprehensive assessment 
and service mandates for special needs learners contained in contemporary 
federal legislation in vocational eduction. During the first project year, 
personnel roles and competencies rel. Hve to operating a CBVA process 
were identified and validated. Initial drafts of the training materials were 
subsequently developed and field-tested in a number of diverse in-service 
and preservice settings during the first and second project years. The third 
and final project year consisted of a series of activities related to analyzing 
field-test data, completing a follow-up of former trainees,refining the train- 
ing products, and conducting product dissemination workshops. Further 
information on the design and the rationale for this CBVA project can be 
found in: 

Albright, L. & Cobb, R. B. (1987). "A model to prepare vocational educators and 
support services personnel in curriculum-based assessment." Interchange, 
7(4), 2-6. Publication of Secondary Transition Intervention Effectiveness 
Institute, University of Illinois, Urbana-Champaign. 

Albright, L. & Cobb, R. B. (1988). "Curriculum-based vocational assessment: A 
concept whose time has come." Journal for Vocational Special Needs 
Education, 10(2), 13-16. 
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Organization and Use of Training System 

1 For whom are these training materials designed? 

This training system consists of a Trainer's Manual and a series of seven 
training modules on the use of curriculum-based assessment procedures 
for students with handicaps in vocational education programs. The train- 
ing system is designed for individuals who are responsible for providing in- 
struction and special services to persons with disabilities in regular 
vocational education settings. Such personnel will vary from district to dis- 
trict, but they are likely to be from among the following groups: vocational 
special needs resource teachers, guidance counselors, vocational and spe- 
cial education instructors, rehabilitation services personnel, vocational 
evaluation specialists, paraprofessionals, and administrators of vocational 
and special education programs. 

2, In what settings can these materials be used? 

The training system can be used in both preservice and in-service set- 
tings. By preservice, we mean traditional, university-based classroom in- 
struction, delivered out of departments of special education, vocational 
education, or vocational rehabilitation. By in-service, we mean a targeted, 
training effort for a select group of personnel from a specific educational 
district or service agency. 

Based on our field-testing in both settings, we have found that the train- 
ing program will have maximum impact when used in an in-service man- 
ner with a team of key personnel from a local education agency. At a 
minimum, the locally-based team should consist of representatives from 
four groups: 

1. Vocational special needs resource teachers or special 
educators responsible for providing support services to 
students with special needs enrolled in regular voca- 
tional settings; 

2. Vocational instructors; 

3. Guidance counselors; and 

4. Administrators of vocational education programs. 

The rationale for a team training approach at the local district level is 
threefold. First, the successful implementation of a curriculum-based voca- 
tional assessment program is contingent upon collaborative leadership ac- 
tivities conducted by each of the key players from the four groups. Second, 
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one of the greatest benefits of a locally-based team effort is increased com- 
munication and understanding between and among the various profes- 
sionals involved in vocational programming for persons with handicaps. 
Although we may assume that such a benefit should naturally occur during 
the day-to-day operations of these professionals, our experience suggested 
otherwise. Hence, a structured and cross-disciplined training program on 
vocational assessment can do much to enhance communication and col- 
laborative program practices among these important professionals. Final- 
ly, the involvement of both direct service providers (i«e«, vocational teachers, 
special needs teachers and counselors) and vocational program ad- 
ministrators is critical to the adoption of a curriculum-based vocational as- 
sessment (CBVA) system within a local district. Since implementation of a 
CBVA process is likely to require important shifts in personnel roles and 
the addition of some resources in a particular district, the active participa- 
tion of key program administrators with direct service personnel ir the lo- 
cally-based training program will increase the likelihood of meaningful 
adoption within the district. 

3. How is the training system organized? 

This training system contains a Trainer's Manual and a series of seven 
training modules. The Trainer's Manual is designed to help the trainer or- 
ganize her/his training effort. The training modules are designed to give 
specific information to the trainees to help them implement assessment ac- 
tivity in their schools and classrooms. 

More specifically, the Trainer's Manual contains a Preface and an Ack- 
nowledgement section, which you have already reviewed. This section of 
the manual follows a question-and-answer format to help you understand 
the organization of the system. The latter part of the manual contains ad- 
ditional information and materials for trainer use in supplementing the 
content within the modules. 

The training modules have been developed to focus on the various as- 
sessment stages and purposes that dictate what type of assessment activity 
to conduct, when it should be done, and by whom. Table 1 shows this con- 
ceptual framework. 

4. What is the content and sequence of each module? 

Table 2 further defines the relationship between each of the training 
modules. You will find that each module follows a common format It begins 

ER?C 
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Table 1: CBVA Implementation Guide 



Key to Personnel 

VSNT = Vocational Special Needs Teacher 
SSE = Secondary Special Educator 
VI = Vocational Instructor 



Purpose 
For 



Timing of 



Personnel 
Involved in 
Assessment 



Assessment 


Assessments 


Activities 


IVTndulp 1 

IflUUUlv A 






Part T» 






Establishing a 


Ongoing 


VSNT 


CBVA Process 




SSE 
VI 
*VA 


Part II: 






Communicat- 


Starts at least one 


VSNT 


T vocational 


year prior to when 


VC 


t vacation 


vocational education 


VI 


program infor- 


is routinely available 


SSE 


mation 


in district 




Module 3 






Placing stu- 


Semester prior to 


*VSNT 


dents in voca- 


when vocational 


VC 


tional 


programs begin 


SSE 


education 




VI 


program 






Module 4 




*VSNT 


Planning a 


During annual IEP 


student's voca- 


review and first 


SSE 


tional educa- 


month of vocational 


VI 


tion program 


class 




Module 5 




VSNT 


Monitoring 


Throughout duration 


student 


of vocational program 


♦ vi 


progress 




SSE 



VC = Vocational Counselor 
VA = Vocational Administrator 
* Suggested Lead Person(s) 

Instruments/ 
Procedures Used In 
Assessment Activities 



Module 6 

Planning tran- 
sitional ser- 
vices 

Module 7 

Evaluating 
CBVA process 



During semester 
prior to graduation 



Ongoing — annually 



VC 

VSNT 
SSE 

yc 

VI 
♦VSNT 

«yc 

VA 
VI 



CBVA Action Plan 



Review process to assure identifica- 
tion of all students with handicaps 
in district and their parents are in- 
formed of options 



Review of student records; inter- 
view with student and parents; as- 
sessments in vocational try-out 
program 



Direct observations of student per- 
formances; inventory of program en- 
vironment and curriculum 



Develop a monitoring system for in- 
dividual students; conduct process 
and product assessments of student 
performance 

Review of student performance; in- 
terview 'vith student and parent 



Annual review process to evaluate 
CBVA processe&s/activities 



Note: Module 2, Understanding Curriculum-Based Vocational Assessment, is omitted from this 
table; however, au OBVA team members will need to carefully review and discuss its con- 
tents. 
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Table 2: CBVA Training Module Sequence 



MODULE 1 

Establishing a 

Curriculum-Based 

Vocational 

Assessment 

Process 



MODULE 2 

Understanding 

Curriculum-Based 

Vocational 

Assessment: 

Purposes and 

Characteristics 



MODULE 3 

Placing 
Students in 
Vocational 
Education 
Programs 



MODULE 4 

Planning a 

Student's 

Vocational 

Education 

Program 



MODULES 

Monitoring 

Student 

Progress 



MODULE 6 

Planning 

Transitional 

Services 



MODULE 7 

Evaluating 
the CBVA 
Process 
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with a goal statement and corresponding competencies. The body of the 
module foil iws, including an introductory section, a strategies or "how-to" 
section, references, and a number of example documents that complement 
the strategies. 

The reference materials section of this manual contains a series of one- 
page abstracts for each of the modules, specifying the goal, assessment 
questions and a brief summary of its content. We have provided these 
abstracts on individual pages s j that they can be reproduced and dis- 
tributed to your trainees to help introduce each module. 

5. Do all trainees need to complete all modules in the training system? 

Referring to Table 1, it can be seen that not all personnel in a district 
will necessarily need training on all of the individual modules. For example, 
a vocational administrator is primarily going to be concerned with estab- 
lishing an assessment process and evaluating its effectiveness. Therefore, 
the first and last nodules in this training system are the most appropriate 
ones for addressing these purposes. 

6. Will all in-service sites require the same scope of training? 

Similarly, each training site will already have in pla:e some or a num- 
ber of the assessment activities elaborated across the entire training sys- 
tem. It is up to the trainer to determine the extent to which some 
assessment activities are already ongoing in a district and customize train- 
ing around those areas in which additional training is needed. 

To determine and respond appropriately to individual participant and 
district training needs will require collaboration and team building. It re- 
quires a systems approach to implementing new or different concepts into 
district educational processes. Module 1 contains a series of steps to guide 
the trainer and trainees through a systems change process of determining 
district needs and planning local efforts to meet those needs. Further, a 
number of outstanding resources are available to the CBVA trainer in the 
politics and processes of systems change. The trainer could refer to Fullan 
(1982), Henshaw, Wilson and Morefield (1987), Squires, Huitt and Segars 
(n.d.), Glatthorn (1987), Mine (1987), Snyder and Giella (1987), and Brody 
(1982) for additional assistance in developing a group process for institut- 
ing change in a school district. For descriptions of systems change ap- 
proaches specific to CBVA, see the publications by the Minnesota 
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Vocational Assessment Guidebook Development Team (1986) and Stodden, 
Ianacone, Boone and Bisconer (1987). 

7. What conditions must exist in a district in order to begin a CBVA train- 
ing effort? 

The difficulty in establishing a CBVA process in a school or school dis- 
trict is likely to vary greatly, depending upon a variety of factors including 
administrative and teacher support, availability of funds, staff expertise, 
and commitment to alternative models of assessment. Based on our field- 
testing of this training program in several diverse settings, there appear 
to be several fundamental requirements that a local district must meet in 
order to begin to adopt a CBVA approach. First, someone must take owner- 
shipof the process in the district. Without afirm advocate for this approach, 
its future is not likely to be great. Second, this advocate must be familiar 
with the district's ongoing assessment processes in vocational education, 
and with the vocational curriculum options in the district. Finally, -he dis- 
trict administration must have made a commitment to expending a 
reasonable amount of resources and staff time to an assessment process. 



8. What additional reference materials are needed to support the use of 
this training system? 

In order to deliver a comprehensive training program, a number of sup- 
port materials has been provided in the Reference section of this Trainer's 
Manual. These include such items as a presentation on validity and 
reliability considerations in testing (Reference B), and a set of discussion 
questions (Reference C), which can enhance comprehension of each train- 
ing module. The bibliography at the end of this document provides you with 
a wealth of sources related to the content of this training program. 



9* What is the next step? 

At this point, you as a trainer should have read each of the eight pre- 
vious organizing questions in this manual, and reviewed the Reference Sec- 
tion material for supportive information. Your next step is now dictated by 
your own training context. 

If you are leading a preservice course (as we have denned it on page 1 of 
this manual), you may want to reduce the emphasis on Part I of Module 1, 
which deals with systems change in a local district, and spend more time with 
Part II of Module 1, dealing with how to communicate vocational training op- 
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tions to students with handicaps and their parents. If, however, you are lead- 
ing an in-service effort, you will find the systems change material in Part I of 
Module 1 very beneficial, both to you and your trainees. 

In either case, you and your trainees should probably be reading and 
discussing the material in Module 2 at the same time you are reviewing 
Module 1. Both modules will give you all a clear idea of how change can be 
accomplished in a local district, how to communicate vocational program 
options to students, and how CBVA can satisfy the vocational assessment 
needs of students, teachers, and support personnel. 
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This section contains a variety of materials that were developed and 
used during the field-testing of this assessment training system. A brief 
description of each is provided, with information on use in preservice or in- 
service contexts. 

Reference A; Abstracts of Each Module 

Description 

This reference provides an abstract of each module, in terms of three 
dimensions: 

1. Module goal; 

2. Key assessment questions addressed; and 

3. Brief summary of module content. 

In-Service/Preservice Use 

These module abstracts should be used in both in-service and preser- 
vice programs to provide participants with an overview of each module in 
the system. The use of these with Table 1 in this manual will provide a com- 
prehensive overview. 

Reference B: Concepts and Terms in 
Educational Assessment 

Description 

This reference provides basic information about five important assess- 
ment concepts and terms: 

1. Vocational constructs; 

2. Validity of instrumentation; 

3. Reliability of instrumentation; 

4. Norm and criterion referencing; and 

5. Types of measurement. 
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Additional sources for obtaining a more extensive treatment of this in- 
formation are cited. A full citation of each source is provided in the Bibliog- 
raphy. 

In-Service/Preservice Use 

This material is present only in the Trainer's Manual; a like version 
does not appear in the training modules. Rather, these concepts and terms 
are embedded in the training content throughout the module series. The 
information contained in Reference B is important for basic trainee under- 
standing of sound assessment practices. What you, as the trainer, will need 
to determine is when to introduce this material and how extensive the 
presentation needs to be. When we introduced this content at the outset of 
the in-service field-test program, it was viewed by the field-test par- 
ticipants as abstract and not especially critical at the beginning of the train- 
ing program. Since these concepts and terms are woven into the training 
material, we recommend, for in-service purposes, that * ou introduce this 
information at various points as the program progresses; attempting to cap- 
ture the "teachable moment" when in-service trainees are ready for this in- 
struction. For example, when discussing data collection methods in Module 
3, the concepts of instrument validity and reliability may be timely for in- 
struction. 

An opposite response to Reference B content commonly occurred in our 
preservice field-testing. That is, the same material that you have in 
Reference B was introduced early in the course and it tended to generate 
a great deal of student interest and discussion. Hence, for preservice pur- 
poses, we recommend that you duplicate Reference B and use it as a course 
reading and discussion piece for at least one session within the first month 
o^the course. Reference E, a sample course syllabus, illustrates when and 
how this information was introduced in one preservice field-test course. 

Finally, listed below are three additional sources which provide greater 
detail to most, if not all, of the material covered in Reference B. 

General Sources 

Borg, W. R. (1987). Applying educational research: A practical guide for teachers 
(2nd ed). New York: Longman. (See, in particular, Chapter 6 on kinds and 
characteristics of educational measures.) 

Vocational Education 

Erickson, R. C, G Wentling, T. L. (1976). Measuring student growth: Techniques 
and procedures for occupational education. Boston: Allyn and Bacon. (See, 
in particular, Chapters 1, 2 & 3 which deal with types of measurements, 
measurement concepts, and criterion-vs. norm-referenced testing. ) 

IB 
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Special Education 

Salvia, J., & Ysseldyke, J. W. (1988). Assessment in special and remedial educa- 
tion (4th ed.). Boston: Houghton Mifflin. (See especially Chapter 2 for basic 
considerations in assessment, and Chapters 7 and 8 for information on 
reliability and validity.) 

Reference C; Module Discussion Questions 
Description 

This reference includes a series of discussion questions for each train- 
ing module. One or more of these questions can be used with the training 
group to generate student discussion of key points within specific modules. 

In-Service/Preservice Use 

During the field- test program, the discussion questions were used only 
in preservice courses. Feedback from training directors indicated that the 
questions were very helpful in stimulating group discussion of module con- 
tent. These questions could, of course, be used in an in-service program. 
However, we believe greater emphasis should be placed on discussions sur- 
rounding the completion of specific in-service assignments, which are very 
applied and oriented to local programming. 

Reference D: Module Assignments for In-Service 
Description 

For each module, one or more assignments are provided. These rein- 
force particular module content and they represent the instruments and 
procedures to use in implementing a CBVA process in the local district. Ex- 
amples of such procedures and instruments are provided in the modules, 
but the assignments engage the participants in adapting these to local 
program needs. 

In*Service/Preservice Use 

The very practical and applied nature of the assignments are especial- 
ly suited for locally-based in-service activity. If participants of a preservice 
course have ready access to local programs, then they may also be used. 
However, given the extensive content to be covered, the three preservice 
assignments in Reference F are more reasonable and appropriate. 
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Reference E: Preservice Course Syllabus 
Description 

The syllabus in Reference E was prepared for a preservice course taught 
at the University of Vermont as part of the field-test program. As shown in 
the course outline, the instructor enhanced the content of the training 
program by incorporating additional source materials. These were used in 
selected topical areas in which the instructor placed particular emphasis. 



In-Service/Preservice Use 

This syllabus is clearly an example for structuring a preservice course. 
The format and content of this syllabus could be adapted for in-service pur- 
poses. However, given the shorter length of training time often present in 
an in-service program and the need to focus on the development and use of 
the assignments in the modules, substantial changes in the syllabus will 
likely be necessary. 

Reference F: Preservice Assignments 

Description 

The three assignments field-tested in preservice courses are provided in 
this reference. The first requires participants to work in teams to examine 
vocational education and training options in a local community. The second 
involves the participant in surveying vocational assessment practices in a local 
education agency. The third preservice assignment is to select a pertinent 
module assignment from among those listed in Reference D. 

In-Service/Preservice Use 

These assignments are directed to a preservice course that will cover 
all of the training content in the module series. They could be adapted tor 
in-service use, especially if the in-service participants are going to complete 
all modules in the training series. 
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ABSTRACT 
Module 1 



Establishing a Curriculum-Based 
Vocational Assessment Process 



GOAL This module has two interrelated goals: 

1 . To present a collaborative team process for 
establishing curriculum-based assess- 
ment practices in local vocational educa- 
tion programs. 

2. To increase participant understanding of 
the full range of vocational education and 
special services available to students with 
handicaps in the local district. 



CONTENT Module 1 consists of two parts. Part I provides 

SUMMARY a series of steps and considerations for local 

team members to use in establishing a CBVA 
process. A major outcome of this activity is the 
development of a local action plan for expand- 
ing and improving assessment services. 

Part II is concerned with procedures for ensur- 
ing that parents, students and educators are 
knowledgeable of the programs and services 
available through vocational education in the 
school district. Strategies for increasing parent, 
student and educator understanding of these 
programs and services are provided. 



KEY ASSESSMENT 1. To what extent are students with hand- 
QUESTIONS icaps, their parents and educators aware 

of and knowledgeable about vocational 
education and special services available in 
our district? 

2. What are the vocational assessment prac- 
tices and needs within our district 7 

3. What changes will be necessary in order to 
successfully establish a CBVA process 
within our school district? 
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ABSTRACT 
Module 2 



GOAL 



Understanding Curriculum-Based Vocational Assessment: 
Purposes and Characteristics 

To increase participant knowledge of: 

1. The central purposes and features of a 
CBVA process; and 

The relationship between a CBVA process 
and the content of this training program. 



2. 



CONTENT 
SUMMARY 



The first section of the module is a description 
of the key elements of CBVA. The second 
tion illustrates how the training program is or- 
ganized around the major purposes of a local 
CBVA effort. 



KEY ASSESSMENT 
QUESTIONS 



1. What is CBVA? 

2. How is the content of the training 
program organized? 
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ABSTRACT 
Module 3 

Placing Students in Vocational Education Programs 

GOAL To assist local personnel in implementing sys- 

tematic CBVA procedures for determining ap- 
propriate vocational education placements for 
students with handicaps. 



CONTENT A series of steps is provided for a local CBVA 

SUMMARY team to follow in helping a student select the 

most appropriate program option in vocational 
education. A variety of assessment for place- 
ment methods is described and illustrated. 



KEY ASSESSMENT 1, Is vocational education the best or most 
QUESTIONS appropriate curriculum for the student? 

2, Which vocational education program is 
most appropriate for the student? 
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ABSTRACT 
Module 4 



Planning the Student's Vocational Education Program 



GOAL 



To increase participant knowledge and skills in 
conducting assessments which identify the spe- 
cial service needs of a student in a vocational 
education program. 



CONTENT 
SUMMARY 



The module begins with a discussion of impor- 
tant differences between assessment for place- 
ment (Module 3) and assessment for program 
planning (Module 4) activities. This is followed 
with guidelines for conducting two types of as- 
sessment for program planning procedures: 

1. A content-specific survey; and 

2. A vocational program inventory. 

The results from these two assessments will ad- 
dress the key questions identified below. 



KEY ASSESSMENT 
QUESTIONS 



1. Is this particular student eligible to 
receive support services in his/her par- 
ticular vocational education program? 

2. In what particular areas of instructional 
remediation should support personnel 
focus to ensure success for a student in 
his/her vocational program? 

3. What is the nature and intensity of sup- 
port services necessary to deliver this in- 
structional remediation? 

4. What criteria should be applied to a 
student's performance which reasonably 
reflects the success norms of the vocation- 
al program? 
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ABSTRACT 
Module 5 



Monitoring Student Progress 



GOAL 



To assist local vocational education and special 
services personnel in establishing coordinated 
procedures for monitoring student perfor- 
mance in vocational education. 



CONTENT 
SUMMARY 



This module focuses on the use of assessment 
and monitoring procedures while the student is 
completing her/his vocational education 
program. Four basic considerations in estab- 
lishing a coordinated monitoring system are 
presented, along with example techniques and 
forms. 



KEY ASSESSMENT 1. Which areas of student performance need- 
QUESTIONS to be monitored? 

2. What are the procedures to be used? 

3. How frequently should student progress 
be measured? 

4. Who is responsible for monitoring student 
progress? 
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ABSTRACT 
Module 6 

Planning Transitional Services 

GOAL To assist local vocational education and special 

services personnel in determining the special 
services needed by a student as she/he exits 
vocational education and enters new employ- 
ment and/or training environments. 

CONTENT The module opens with a brief discussion of the 

SUMMARY rationale behind the transitional services in- 

itiative for individuals with handicaps. The 
various options for transitional service delivery 
are subsequently described, as are the assess- 
ments which need to occur in planning and 
monitoring a student's ransiticn. A case study 
of one student is presented to illustrate the 
types of assessment activities that take place 
during the transitional period. 



KEY ASSESSMENT 1. In which outcomes (either postsecondary 
QUESTIONS education, training or employment) will 

the student be functioning upon gradua- 
tion? 

2. What services will the student require at 
his/her selected option? 

3. Who will be responsible for monj^ring the 
student as she/he transitions into the new 
environment? 

4. Now effective are the special services in 
helping the student succeed in the new 
environment? 
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ABSTRACT 
Module 7 

Evaluating the Currici.lum-Based 
Vocational Assessment Process 

GOAL To assist local personnel in evaluating the ef- 

fectiveness of the CBVA process in determining 
appropriate vocational education and special 
services for students with handicaps. 



CONTENT A series of steps is provided for organizing, con- 

SUMMARY ducting and reporting an evaluation of a local- 

ly-based CBVA process. A team approach to 
this evaluation is advocated. Sample instru- 
ments and forms are also provided for team 
member use. 



KEY ASSESSMENT 1. How effectively is the CBVA process work- 
QUESTIONS ing in our district? 

2. What improvements are needed to 
strengthen the CBVA process? 



Reference B 
Concepts and Terms in 
Educational Assessment 1 

There are several concepts and terms in educational assessment that 
arc important to understand in order to develop and implement assessment 
practices upon which school districts can rely. These can be categorized into 
five areas: 



1. 


Vocational constructs (interest, achievement, and ap- 




titude); 


2. 


Validity of instrumentation; 


3. 


Reliability of instrumentation; 


4. 


Referencing (norm and criterion); and 


5. 


Types of measurement (direct/indirect and 




process/product). 



Each of these assessment areas will be discussed ii) turn. 

Vocational Constructs 

Vocational constructs are ideas or concepts that we are trying to 
measure to assist in making decisions about programming for students with 
handicaps. They can be grouped into three areas: 

1. Vocational interests; 

2. Vocational achievement; and 

3. Vocational aptitude. 

Vocational Interests 

Simply put, vocational interests are occupational cluster areas in which 
a student has an interest. While easy to understand, vocational interest as- 
sessment for high school age adolescents with handicaps is extremely dif- 
ficult to measure. Our instrumentation is simply not well enough developed 
to be able to determine enduring vocational interests in young people who 
have typically had very limited and even distorted experiences with the 
world of work. This problem is further compounded in vocational education 

*Note to Trainer: The citations presented within this Reference B paper are con- 
tained in the Bibliography section of this manual. 

.70 
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in that a student may have a strong interest in a particular vocational 
program for reasons that are not measurable with traditional vocational 
interest inventories (i.e., the student's friends are enrolling in that 
program, the program fits into the student's schedule, or the teacher h s a 
reputation of being a great teacher). With CBVA, vocational interest as- 
sessment is largely oriented around providing students with a clear idea of 
what vocational program options are available and what the requirements 
are for entry and exit from each program. 

Vocational Achievement 

Vocational achievement is a statement of skills that a student can 
demonstrate at the time of testing. No attempt to predict future vocation- 
al ability can be inferred from vocational achievement assessment. With 
CBVA, vocational achievement assessment is focused on measuring what 
skills a student can demonstrate prior to, during, and upon completion of 
a vocational course of study. 

Vocational Aptitude 

Vocational aptitudes are inherent skills a student is purported to pos- 
sess that are germane to differing occupations. Skills like spatial ability, 
numerical and verbal reasoning, language usage, mechanical and abstract 
reasoning, and even spelling are commonly measured vocational aptitudes. 
The logic of vocational aptitude assessment is one of measuring student ap- 
titudes with the intent of predicting a "fit" with occupations requiring 
similar traits. Our ability to accurately measure vocational aptitudes of 
adolescents with handicaps is extremely limited and problems exist that 
are similar to those with vocational interest assessment. Thus, since we 
have no valid way of measuring aptitudes for vocational education program 
options, aptitude measurement in CBVA is eliminated. 

Validity of Instrumentation 

Validity is relatively easy to define but difficult to put into practice. 
Simply put, validity of instrumentation means that a test measures what 
it states it is designed to measure, and that the results of that test can help 
make decisions about the individuals who have taken it. There are five dif- 
ferent types of validity: 

1. Face validity; 

3 1 
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2. Construct validity; 

3. Predictive validity; 

4. Content validity; and 

5. Concurrent validity. 

It is important to know each type of validity since some vocational as- 
sessment instruments will rate very highly on one form of validity and poor- 
ly on others. It is also important since some types of validity are more 
important than others in vocational education assessment. 

Face Validity 

A test has face validity if it appears to measure what it purports to 
measure. It is the least important type of validity, because a test may look 
good, but not really measure what it is designed to. Commercial work sam- 
pling systems have high face validity, in that they look like they are measur- 
ing vocational skills that are important to differing jobs. However, the skills 
measured in a commercial work sample frequently have little or no relation- 
ship to the most important skills in vocational programs for which the work 
sample is purported to measure. Thus, while they have face validity, they 
frequently yield little usable vocational interest or achievement data, 

Construct Validity 

Construct validity refers to the extent to which a test measures a 
theoretical trait, attribute, or construct (i.e., specific vocational interests or 
aptitudes). While conceptually important, construct validity has little place 
in curriculum-based vocational assessment, since CBVA does not typically 
involve the measurement of traits or attributes. 

Predictive Validity 

The results of a test have predictive validity to the extent that they help 
predict how a person will function or perform skills on some future task. 
The only way to measure predictive validity is to keep careful track of the 
manner in which results of assessments yield predictably valid results at 
some future time. While CBVA instruments are not typically concerned 
with predictive validity, two specific? instances need mentioning. 

First, it will be important for assessment personnel to examine the 
predictive validity of instruments used to assist in making placements of 
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students with handicaps in different vocational programs. This would be 
accomplished by monitoring the number of students who decide to change 
their vocational program focus, the reasons for those changes, and the ex- 
tent to which instrumentation used to assist in program placement 
decisions should have helped to avoid the need for such changes. Clearly 
the goal here would be to find the instruments that minimize the mis- 
matches in progress selection. 

Second, a similar issue would arise in the selection and use of in- 
strumentation designed to predict the need for transition services for stu- 
dents who are exiting vocational programs. 

Content Validity 

Content validity refers to how adequately a test covers the entire range 
of subject matter about which the test is designed to measure. For example, 
if you are trying to determine vocational program interest of students, the 
instruments used to do that would have high content validity only to the 
extent that they included all vocational program options, and provided ac- 
curate, detailed information about the requirement, of each of these 
program options. Content validity is an extremely important consideration 
in CBVA since sound instructional decision-making about students can 
only be accomplished through instruments that measure the entire scope 
of program content. 

Concurrent Validity 

Concurrent validity exists if a test's results accurately portray current 
student performance. For example, if a student performed poorly on a cos- 
metology work sample, yet was functioning perfectly adequately in a cos- 
metology vocational program, the test would have poor concurrent validity. 
Concurrent validity is also extremely important in CBVA, primarily in 
program planning and monitoring functions. It is usually determined by 
matching a student's test performance with other measures of the same 
performance and looking for similarities in results. 

Reliability of Instrumentation 

Reliability of assessment instruments is concerned with the stability of 
a test's measurement power. It reflects confidence we have in accepting and 



9 

ERIC 



33 



Trainer's Manual 

Currlculurn-Based Vocational Assessment 



33 



generalizing test results. There are four ways of testing the reliability of a 
measurement device: 

1. Test-retest reliability; 

2. Split-halfreliability; 

3. KR 20 reliability; and 

4. Interrater reliability. 

While the actual formulas for using these types of reliability will not be 
provided here, the reader is encouraged to consider determining the 
reliability of his/her informal tests. The formulas are very simple to use and 
can be found in any good measurement text (e.g. Salvia & Ysseldyke, 1988, 
pp. 110-118). 

Test-Retest Reliability 

The first type of reliability is called test-retest reliability which is deter- 
mined by having a student take the same test twice, or take an alternate 
form of the same test, with approximately two weeks or less between test- 
ing with no training on test content. If the scores are approximately the 
same, the test can be considered reliable and you may have confidence that 
it is accurately measuring a stable construct. You should consider using 
this type of reliability if you have a test that you can use repeatedly (e.g. a 
pre/post-test) to measure improvement in students. This is because you 
want to be certain that changes in test results truly reflect changes in stu- 
dent ability rather than simply an artifact of how the test was made. 

Split-Half and KR 20 Relia bility 

Sometimes we may put a test together in which all test items are 
designed to measure the same skill or student characteristics. Theoretical- 
ly, then, each student taking the test would have the same likelihood of 
answering any single test question correctly as any other test item. This 
form of reliability is called internal consistency and can be determined 
through a split-half or KR-20 reliability assessment. These forms of 
reliability are extremely easy to calculate and would be used by a special 
needs teacher or vocational instructor with tests that were only going to be 
taken once by a student. An example might be a test used for program plan- 
ning in the area of fractions or vocational textbook comprehension. Again, 
if the reliability index is fairly high (.80 or higher) you can be certain that 
each item in the overall test is measuring the same skill. Conversely, if the 
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index is low, it is likely that there are some items that are measuring some- 
thing you do not intend to measure, and the test questions should be 
analyzed to locate those which are consistently discrepant from all others. 

Interrater Reliability 

Interrater reliability involves two individuals administering, scoring, 
and interpreting the same test to the same person, and determining if the 
results are the same. You are looking for a high percentage of agreement 
between the two raters which would signify that you are accurately measur- 
ing the same skill or behavior. Interrater reliability is most commonly used 
with observation measurement or with tests or rating scales whose results 
may be subjectively interpreted. 

Referencing 

There are two ways to interpret or make sense of the results of an as- 
sessment. The results of an individual's test can be interpreted or compared 
with the results of another group of people who took the test (norm referenc- 
ing). Or, the results can be compared against a standard of performance 
(criterion referencing). 

Generally, norm referenced tests are associated with formal, stand- 
ardized assessments, and are not appropriately included in CBVA since 
comparing one person's test results with his/her peers provides little help 
in instructional decision-making. An exception would be the establishment 
of local norms which might be helpful in determining if a handicapped stu- 
dent was eligible for special services in vocational education. However, most 
CBVA instrumentation will be criterion referenced, since most vocational 
educators are familiar with performance standards and establishing in- 
structional decision-making around those standards. 

Types of Measurement 

The last concepts in assessment that need discussion are the notions of 
direct and indirect, and formal and informal types of measurement. 

Generally speaking, those who use CBVA will want to utilize assess- 
ment instruments that are direct rather than indirect. This means that in- 
structional decisions should be made by assessing a student's performance 
on tasks that are directly relevant to vocational program content. For ex- 
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ample, the decision to provide remedial help in mathematics to a student 
in a mar 1 ine trades program should be made by assessing a student on a 
sample of math taken directly from the machine trades curriculum, rather 
than through using his/her results from a math sub-sample of a stand- 
ardized achievement test. 

Similarly, informal rather than formal types of assessment should be 
used most extensively with CBVA. Informal tests are generally teacher- 
made and more reflective of local norms and curriculum content than are 
formal tests. Thus, they have greater content validity and can usually be 
used more efficiently than can more formal, standardized assessments. 
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DISCUSSION QUESTIONS 
Module 1 

1. In order to implement a CBVA process in a school district, changes in 
present practices are likely to be needed. Identify what these changes 
are likely to be, and how they may best be changed to ensure that the 
CBVA process becomes established in the district or agency. Con- 
siderations relative to personnel, attitudes and cost factors should be 
included in this discussion. 

2. Identify and discuss the basic types of information that parents and 
students need about vocational education programs and special ser- 
vices. 

3. Of the various methods suggested for providing information to parents 
and students, select two that are likely to be the most effective. Also, 
discuss the reason(s) for selecting each method. 
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DISCUSSION QUESTIONS 
Module 2 

1. According to the proponents of CBVA, assessment is viewed as a con- 
tinuous, ongoing process. How does this compare with your view of as- 
sessment for students in vocational education? 

2. To what extent is CBVA a realistic approach for use in your district? 
Consider such factors as personnel and time requirements and how 
the characteristics of CBVA match with present practices in your dis- 
trict. 

3. What would appear to be the major changes needed in order to imple- 
ment a CBV A prccess in your district? 

4. Part II of Module 2 identifies and briefly describes the seven modules 
in the CBVA system. Of these seven, select two that are of highest 
priority to y ou and discuss the reasons for your selections. 
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DISCUSSION QUESTIONS 
Module 3 

1 . Three personnel are identified as playing key roles in the assessment 
for placement process: 

A. The special educator in charge of the student's IEP; 

B . A vocational guidance counselor; and 

C . The individual who provides special services to students 
with handicaps in vocational education programs. 

Discuss the roles of each in the assessment for placement process and suggest 
other personnel who could contribute to this process. 

2. A variety of informal methods for assessing students in the placement 
stage has been described in this module, such as: 

A. Cumulative records review; 

B . Student and parent interviews ; and 

C. Vocational tryouts. 

Discuss each of these methods and determine which one(s) would likely produce 
the most useful information for placing a student in a vocational program. 
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DISCUSSION QUESTIONS 
Module 4 

1 . Discuss the major differences between assessments conducted for stu- 
dent placement (Module 3) and assessments for student planning 
(Module 4). 

2. When assessing for the purpose of planning a student's vocational 
program, how will the assessment data be used in the vocational IEP 
process? 

3. Who should have the primary responsibility for conducting assess- 
ments during the program planning stage? When should these assess- 
ments occur? 

4. Both content-specific surveys and vocational program inventories 
were recommended in this module. Discuss how the two methods dif- 
fer in the information they yield and why both types of information 
are important during the program planning stage. 
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DISCUSSION QUESTIONS 
Module 5 

1. Discuss the roles of the vocational special needs teacher and the voca- 
tional instructor in the monitoring process. 

2. Two assessment and monitoring methods discussed in this module are 
direct observation and knowledge testing. Discuss the similarities and 
differences between these two methods. 

3. Discuss the advantages and disadvantages of using "aimlines" in 
monitoring student progress. 
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DISCUSSION QUESTIONS 
Module 6 

1. Discuss the types of student assessment information that need to be 
used in planning transition services for a student. Consider the voca- 
tional instructor, the vocational special needs teacher and the 
student's IEP manager as information sources in this discussion. Be 
as specific as possible in identifying the types of assessment informa- 
tion needed from each source. 

2. From among the various personnel involved in the student's vocation- 
al education program, identify and discuss which one(s) should assume 
responsibility for planning and monitoring the student's transitional 
program. 

3. While the focus on transitional services becomes important near the 
end of the student's vocational program, when should planning for the 
student's transition from school to employment begin? 
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DISCUSSION QUESTIONS 
Modulo 7 

1. One important principle in the evaluation of educational programs is 
to identify and involve the various audiences associated with the par- 
ticular program being evaluated. Identify the various audiences in- 
volved in or affected by a CBVA process and discuss how these 
audiences should be involved in an evaluation of a CBVA process. 

2. One of the first steps in designing or "focusing" on evaluation is to 
determine the key questions that need to be addressed. Discuss the 
importance of this activity and suggest several key questions that 
should be asked in an evaluation of a CBVA process. 

3. Discuss the importance of a team approach to evaluating a CBVA 
process and suggest the personnel, by position title, that should be 
members of a CBVA evaluation team. 

4. Each evaluation team must decide how much evaluation information 
should be collected. Outline and discuss the kinds of information 
needed in a CBVA evaluation and the methods to use for obtaining 
this information. 
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IN-SERVICE ASSIGNMENT 
Module 1 

Listed below are two assignments. The first pertains to Part I of the 
module, "Teaming to Implement a CBVA Process". The second is based 
around Part II of the module, "Communicating Program Options in Voca- 
tional Education." 

1. As a team, complete each of the products indicated in the steps in the 
strategies section of Part I. These are: 

Step 1: Prepare a mission statement. 

Step 2: Prepare a problem statement. 

Step 3: Develop a prioritized list of needs statements. 

Step 4: Produce a paper outlining the CBVA relationship to 
problem statement and needs statement. 

Step 5: Identify goals and objectives. 

Step 6: Prepare action plans for each objective. 

Step 7: Identify constraints and incentives. 

Step 8: Revise action plans, following field-testing. 

2. As a team, and in a group session, identify the vocational education 
programs and special services available to students with handicaps. 
Also, discuss the methods presently used in the district to inform these 
students and their parents of the options in vocational education. 
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IN-SERVICE ASSIGNMENT 
Module 2 

1. As a group, discuss the major features of the CBVA process. In advance 
of this discussion, each feature discussed in Module 2 should be divided 
among participants and they be charged with leading group discus- 
sion on the respective feature. 

2. After receiving an overview of the CBVA module sequence and examin- 
ing the content within each module, the group is to use Table 3, CBVA 
Implementation Guide , in determining which particular personnel, by 
name and position title, should be involved in system assessment ac- 
tivities. This adaptation would occur in the third column of Table 3 
(Personnel Involved in Assessment Activities). 
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IN-SERVICE ASSIGNMENT 
Module 3 

Listed below are several team and individual activities. Review each in 
a group training session, determine which of these activities are most 
needed in your district and proceed accordingly. 

For each activity completed, prepare a brief report explaining what you 
did, how you did it and how effective it was. Please attach a copy of instru- 
ments/products used to your summary report. 

1. Using the assessment and planning procedures stated in the module, 
design and conduct a team process for determining the vocational 
placement of one handicapped student. 

2. Using the procedures and reporting format in Examp le Document #1 , 
conduct a review of one student's cumulative record. 

3. Adapt the Intake Interview form shown in Example Doc ument #2 to 
your information needs and use it with two students. 

4. Using Example Document #5 as a guide, organize a performance 
sample for one vocational program and try it out with two students, 
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IN-SERVICE ASSIGNMENT 
Module 4 

Listed below are two team activities. Determine which one of the two is 
most needed and implement it. Prepare a two to three page summary report 
explaining what you did, how you did it, and how effective it was. Please 
attach a copy of the instruments used to your summary report. 

1. Pick out an area of importance »toss many vocational programs (e.g., 
math, reading, safety), and develop a content-specific survey. Try it 
out on a student and generate a series of teaching objectives. 

2. Example Document #1 (Vocational Program Inventory) is to be used in 
determining the organization and structure of a vocational program. 
Try this form out in a vocational program, and match a particular 
student's skills with the results of the Inventory. Use this information 
to generate instructional support areas for the vocational component 
of the IEP. 
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IN-SERVICE ASSIGNMENT 
Module 5 

Team/Group Activity 

1. Within the strategies section of this module are four basic considera- 
tions in establishing a coordinated student monitoring system. These 
are: 

A. Specify student assessment/monitoring areas; 

B . Determine assessment procedures ; 

C. Determine the frequency of assessments; and 

D. Identify personnel responsibilities. 

Use these considerations in setting up a progress monitoring system 
for a student in a specific vocational education program. 

Individual Activity 

2. For one student who has a designated special services plan (Module 
4) t determine specific areas in which student progress will be 
monitored in the vocational education program. 

3. Adapt or design a rating scale or a checklist for observing student per- 
formance on a particular task important in a vocational program. Try 
it out with one student and report on its usefulness. 

4. Using the procedures and examples within this module, design an aim- 
line for one student and use it in tracking her/his progress over time. 

5. Adapt the sample service delivery report in Table 4 or select a similar 
instrument and use it in monitoring the special services provided to 
one student in a vocational program. 
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IN-SERVICE ASSIGNMENT 
Module 6 

Complete one of the two activities below which would best assist you 
and your school district in developing a joint vocational/special education 
transitional services effort. 

1. As a team, select two or three handicapped vocational education stu- 
dents who are scheduled to graduate at the end of the school year, and 
collaboratively develop an ITP. 

2 . Identify a sample of 10 to 15 former vocational education students who 
were on IEPs, and conduct a brief, informal student follow-up examin- 
ing their perceptions of the quality of their participation for the adult 
world, and the quality of any adult services they have received since 
graduation. 
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IN-SERVICE ASSIGNMENT 
Module 7 

Listed below are three evaluation activities. After reviewing this list 
and consulting with your instructor, select and complete one of these ac- 
tivities. 

1. Complete the first two steps listed in the procedures section of this 
module. The resulting product of this activity should be a plan for 
evaluating CB VA in your district. This plan should outline the following 
elements: 

A. Key questions or items to be addressed in the evaluation. 

B . Identification of sources of evaluation information. 

C. Instruments to be used to collect necessary information. 

D. Personnel (by position) who should be involved in organizing 
and conducting the evaluation in your district. 

2. Using the instrument development procedures suggested in Table 2 of 
this module, prepare an instrument for use in evaluating the CBVA 
process in your district. Submit this instrument to your instructor 
along with a brief summary of the procedures used in developing this 
instrument. 

3. Examine the systems or approaches presently being used in your dis- 
trict to evaluate special education and vocational education programs. 
After examining each, determine how an evaluation of aCBVA process 
could be integrated into one of the existing systems. Prepare a report 
which summarizes the structure of this evaluation approach and 
shows how the CBVA process could become part of this approach. Or- 
ganize this report as a proposal to an administrator in your district for 
accomplishing this change. 
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COURSE TITLE: 



Curriculum-Based Vocational Assessment 
for Students with Special Needs 



COURSE NUMBERS: 



EDSP 295/VOTC 295 



INSTRUCTOR: 



Brian Cobb, Assistant Professor, Depart- 
ment of Special Education, 405 A Waterman 
Building, University of Vermont, Bur- 
lington, VT 05405; Tel. 656-1351 



CREDITS: 



Three 



COURSE 
DATES/TIMES: 



Tuesdays: 4:30-7:30 p.m.; January 26-May 
10, 1988; 409 Waterman Building 



REQUIRED 
READINGS: 



Selected articles, chapters, and a series of 
training modules assigned by instructor 



Student Competencies 

By successfully completing this course, the student will: 

1. Demonstrate knowledge of the processes and assumptions under- 
lying curriculum-based vocational assessment and how they are 
suited to implement the assessment mandates in the Carl Perkins 
Vocational Education Act of 1984; 

2. Demonstrate an understanding of the major instruments needed to 
conduct curriculum based vocational assessment and how to inter- 
pret results within the context of regular vocational education 
programs; 

3. Demonstrate knowledge of the steps needed to establish a sys- 
tematic curriculum based vocational assessment process in the 



4. Demonstrate an understanding of the similarities and differences 
between traditional vocational evaluation and curriculum-based 
vocational assessment; 

5. Develop a local plan to implement curriculum-based vocational as- 
sessment in his/her school; and 

6. Implement various types of curriculum-based vocational assess- 
ment in educational and training environments. 



schools; 
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Syllabus Page 2 
EDSP/VOTC 295 


Major Course Requirements 


1. 


Pre-service Assignment #1 (Team Activity) 


2. 


Pre-service Assignment #2 (Individual Activity) 


3. 


Pre-service Assignment #3 (Individual Activity) 


4. 


CBVA Module Evaluations (7) (Individual Activity) 


5. 


Discussion questions 


Requirements Explanation 


1. 


Pre-service Assignment #1: This assignment is a team activity to 
be completed by teams of classroom participants ranging in size 
from one to three individuals . Specific directions for completing this 
assignment can be found on pp. 20-22 of CBVA Module 1. This as- 
signment is due on February 23. Criteria to L~ used by the instruc- 
tor to evaluate the quality of this assignment can be found on the 
Product Assessment Checklist that is a part of CBVA Module 1. 


2. 


Pre-service Assignment #2: This assignment is to be completed in- 
dividually by each class participant. Directions for completing this 
assignment can be found on page 23 of CBVA Module 1. This as- 
signment is due on May 10, and will be evaluated by the instructor 
through the use of the s r . < Product Assessment Checklist dis- 
cussed for Assignment #1 


3. 


Pre-service Assignment #3: CBVA Modules 2- 7 each contain one or 
more pre-service assignments) at the very end of the module. 
Based on your present needs and interests, as well as those of your 
agency or school, select one module and complete the pre-service 
assignments). Again, the instruct ns for completirrr the assign- 
ment and the evaluation criteria to be used to evaluate its quality 
are contained in each module. This assignment is due on May 10. 


4. 


CBVA Module Evaluation (7): Each of the seven modules you will 
read in this course contains an evaluation form which you must 
complete. These forms will then be mailed to the developers of the 
CBVA training system and will help them to improve the system. 
F lease be complete and thorough in your evaluation comments. 
Each module evaluation is due the week afte* it has been assigned 
as a reading (e.g. Module 1 evaluation is due February 9). 
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5. Discussion Questions : Approximately 3 - 4 times during the 
semester, you will be required to lead a discussion around the topics 
contained in the Discussion Questions section of each of the CBVA 
modules. The instructor will assign you your particular discussion 
question. In addition to leading the discussion, you must turn in a 
1-2 page typed narrative highlighting the important points you in- 



tend to make as you lead the discussion. 
Grading 

1. Pre-service Assignment #1 10% 

2. Pre-service Assignment #2 15% 

3. Pre-service Assignment #3 15% 

4. CBVA Module Evaluations 15% 

5. Discussion Questions 15% 

6. Attendance/timely completion of assignments 10% 

7. Participation/ discussion 10% 

8. Final Examination 10% 



100% 

Grading System 

The standard University of Vermont grading system will be utilized. 
More specifically, the following point distributions will determine final 
grades: 

A+ = 97 - 100 B+ = 87 - 89 C+ = 77 - 79 

A = 94 - 96 B = 84 - 86 C = 74 ■ 76 

A- = 90 - 93 B- a 80 - 83 C = 70 - 73 

Recognition for accuracy, meeting required timelines, and profes- 
sionai appearance of products will all be considered in determination of 
grades for this course. Students are expected to complete all assigned 
readings and actively participate in each class. 
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Schedule of Seminars 



Class # 



Date 



Topics 



1/26 Registration 
Introductions 

Overview of course and syllabus 
Review of Perkins assessment mandate 
Discussion of characteristics and 

assumptions of curriculum-based 

assessment 
Readings: Cobb & Larkin (1985) 
Deno(1985) 
Albright & Cobb (1987) 
2/2 Review of curriculum-based assessment 

technology and relationship to 

vocational education 

Review of CBVA and differences from 
traditional vocational evaluation 

Review of purposes and questions of 
CBVA 

Readings: Albright & Cobb (Interchange, 1987) 

Ianacone & Leconte (1986) 

CBVA Module #2 
2/9 Discussion of CBVA Module System 

Discussion of technical concepts in 
assessment (overview) 

Individual presentations of discussion 
questions 

Readings: Carmines & Zeller (1979) 

"Chapters 2 & 4" 
2/16 Finish individual presentations 

of discussion questions 
Continue of discussion and exercises 

on validity and reliability 

assessment 
Readings: CBVA Module #2 
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2/23 


Finish discussion and exercises on 
reliability . nd validity assessment 

Individual presentations of discussion 
questions 




Readings: 


CBVA Module #3 

Chapter 4 - Qualitative Interviewing 
Chapter 7 - In-depth Interviewing 


6 


3/1 


Presentation on interviewing techniques 
Individual presentations of discussion 
questions 

Discussion of Cornell Test of Basic Skills 
and Generalizeable Skills Tests 




Readings: 


Greenan (1986) 
Hill (1986) 


7 


3/8 


Continued individual presentations of 

discussion questions 
Exercises with Cornell Test of Basic 

Skills 




Readings: 


CBVA Module M 




3/15 


NO CLASS - SPRING RECESS 


8 


3/22 


Presentation on informal assessment and 

ecological assessment 
Exercises with ecological assessment 

instrumentation 

Individual presentations of discussion 
questions 




Readings: 


Handouts on checklist and rating scale 
development 


9 


3/29 


Continued individual presentations of 
discussion questions 

Exercises with checklists and rating 
scales 




Readings: 


CBVA Module #5 
Blankenship(1985) 
Rosenberg & Sindelar (1983) 
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10 


4/5 


Presentation on direct observation techniques 






Individual presentations of discussion 






questions 






Exercises with direct observation 






techniques 




Tl 1 • 

Readings: 


Deno&Mirkin(1977) 






Vasa (1980) 


11 


4/12 


Presentation on data-based instructional 






decisionmaking 






Exercise on data-based instructional 






decision making 






Discussion on sradincr and evaluation 






procedures 




Headings. 


Lro VAMOuUle no 






Gaylord-Ross (1986) 






TT_ J I «T 1 • • 1 1 J f 1 r^i • <•<• 

Handout - Individual and Job Skills 






Si 1 Yf\ tY\ Q 1*1 A. a 


12 


4/1 Q 


ire sen ia iion on assessing transitional 






sprvipps 






inuiviuuai presentation 01 discussion 






Questions 

\A O VAX/HO 






xr reociiittiiun on use oi individual and oou 






Skills Summaries 




IVCClUllAuiJt 


CRVA Madulp #7 






/VLDIlgnt ^ILfOOj 
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i reseniaiion on program evaluation 






individual presentations oi discussion 






questions 




Readings: 


None 


14 


5/3 


Presentations - "Pre-service 






Assignment #1" 






Review of course content for final 






examination 




Readings: 


Review for final examination 


15 


5/10 


Final examination 






Course Evaluations 






1 .. 



Syllabus Readings 



Albright, L. (1986). "Program evaluation and generalizable skills instruction" 
Journal for Vocational Special Needs Education, 9(1) 39-42. 

Albright, L., & Cobb, R. B. ( 1987). "A model to prepare vocational educators and 
support services personnel in curriculum-based assessment. Interchange, 
7(4), 2-6. 

Albright, L., & Cobb, R. B. (1988). "Curriculum-based vocational assessment: A 
concept whose time has come* Journal for Vocational Special Needs 
Education, 10(2), 13-16, 

Blankenship, C. S. (1985). "Using curriculum-based assessment data to make 
instructional decisions." Exceptional Children, 62(3) 233-238. 

Carmines, E. G., & Zeller, R. A. (1979). Reliability and Vt 'dity Assessment 
(Sage University paper series on Quantitative Applications in the Social 
Sciences #07-017). Beverly Hills, CA: Sage Publications. 

Cobb, R. B. , E Larkin, D. (1985). "Assessment and placement of handicapped 
pupils into secondary vocational education programs." Focus on Excep- 
tional Children, 17(7), 1-14. 

Deno, S. (1985). "Curriculum -based measurement: The emerging alternative." 
Exceptional Children, 52(3), 219-232. 

Deno, S. , & Mirkin, P. (1977). Data-based program modification: A manual 

Gaylord-Ross, R. (1986). "The role of assessment in transitional, supported 
employment." Career Development for Exceptional Individuals, 9(2), 129- 
134. 

Greenan, J. P. (1986). "Curriculum and assessment in generalizable skills in- 
struction." Journal for Vocational Special Needs Education, 9(1), 3-10. 

Ianacone, R. N., & Leconte, P. J. (1986). "Curriculum-based vocational assess- 
ment: A viable response to a school-based service delivery issue." Career 
Development for Exceptional Individuals, 9(2), 113-120. 

Patton, M. Q. (1980). Qualitative evaluation methods. Beverly Hills, CA: Sage 
Publications. 

Vasa, S. F. (1980, February). Alternative procedures for grading handicapped 
students in the secondary schools. (Unpublished manuscript). 
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PRESERVICE ASSIGNMENT #1 : 
Team Activity 

In order to perform curriculum-based vocational assessment processes, 
one must be familiar with the various curricula on which these processes 
will be based. "Curriculum," in this assignment, includes the vocational 
training and specialized service options available to disabled students at 
both secondary and post-secondary levels. The purpose of this team assign- 
ment is to explore the range of vocational education and training programs 
and supporting specialized services in your community ac? oss three objec- 
tives: 

Objective 1: The curricular offerings (programs at secondary and 
post-secondary levels; 

Objective 2 : The eligibility criteria entry-level skills, and specialized 
supportive services associated with entry into and on- 
going maintenance in these offerings; and 

Objective 3 : The procedures used to communicate information about 
these programs and services to disabled individuals 
and/or their parents or guardian. 

More detailed information about completing this team activity can be 

found on the attached instruction sheets. N.B. It is recommended that the 

team consist, as much as possible, of individuals from the same service 

delivery area. Team size, however, must be limited to no more than three 

members. 

Instruction Sheet for Assignment #1 

Step 1: As a team, identify the names of the agencies which exist in the 
following settings: 

• Secondary vocational programs at regional centers; 

• Secondary vocational programs at high schools; 

• Community colleges; 

• Area vocational-technical institutes ; 

• Proprietary schools; 

• Public training programs for adults (i.e., JTPA, DD/MH/MR); 

• Sujported/transitional employment programs; 

• Apprenticeship programs in public and private settings; 
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• Rehabilitation centers/programs; 

• Others. 

Step 2: As a team, determine which team members will be responsible 
for examining specific agencies identified in Step 1. Each team 
member must examine an agency in no less than two settings 
noted in Step 1. For example, a secondary vocational specialneeds 
resource teacher might assume responsibility for examining 
his/her own agency, as well a community college or rehabilita- 
tion program setting. 

Step 3: Using the Reporting Guide that appears on the following pages, 
review the specific programs and services within each agency. 

Step 4: Assemble individual team member Reporting Guides into a single 
group report. 

Step 5: As a team, organize a 15 minute class presentation summarizing 
the major findings of this assignment. In preparing this presen- 
tation, address questions such as the following: 

A. What is diversity of programs and services? (e.g., number of 
program types, age range and severity of disabling condition 
of persons served); 

B. How specific are eligibility criteria for vocational/employ- 
ment program entry? 

C. How extensive are specialized services in differing 
programs? 

D. How thorough are procedures for communicating informa- 
tion about these programs to potential disabled participants? 
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Reporting Guide 

Team Member 

Name: 

Agency Name and 

Address: 



Agency Contact Person: 

(Position) 

(Telephone No.) . 

List (or attach a listing) of: 

1. Specific programs offered; 

2. Eligibility criteria for these programs and any other 
prerequisite skills for entry into these programs; 

3. Specialized support services (e.g., counseling, tutorial, 
interpreter) to assist disabled individuals succeed in 
these programs; and 

4. Procedures used to communicate information about the 
programs and services to disabled individuals, and/or 
their parents/guardians. 
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PRESERVICE ASSIGNMENT #2 
Individual Activity 

Survey of Vocational Assessment Practices 
in Your Agency 

The purpose of this assignment is to examine vocational assessment 
practices in your agency. This review is to be completed as you move 
through Modules 3-6 of the "Curriculum-Based Vocational Assessment 
Training Package." Complete a 1-2 page narrative around each of the fol- 
lowing survey questions: 

1 . How are individuals with handicaps presently assessed 
for placement into vocational education or employment 
training programs in your agency? 

2 . What are the assessment practices presently being used 
to plan the nature and intensity of supportive services 
for individuals with handicaps in those programs? 

3 . How is the progress of students with handicaps assessed 
(or monitored) as they move through these programs? 

4. As students near completion of their programs, how is 
the need for transitional support services in subsequent 
training and employment settings assessed? 
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PRESERVICE ASSIGNMENT #3 
Individual Activity 

1. Your course instructor has a listing of assignments for Modules 1-7 
(Reference D in Trainer's Manual). Based upon your present profes- 
sional position, select one module appropriate for your involvement in 
CBVA and complete one of the assignments indicated for this module. 
For example, if you are presently a vocational counselor, you may tar- 
get Part II of Module 2, Module 3 or Module 7 as your area for com- 
pleting an assignment. 
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Module 1 

Establishing a CBVA Process 

GOAL: Upon completion of this module, you will have 

an understanding of: 

1. The steps and considerations necessary to 
establish a curriculum-based vocational 
assessment (CBVA) process; and 

2. How best to communicate the full range of 
vocational education programs and special 
services available to students with hand- 
icaps in your school district. 



COMPETENCIES: To establish a CBVA process in a local setting, 

you should demonstrate: 

1. Knowledge of the major components of a 
CBVA process and how these components 
interrelate; 

2. Skills in identifying and analyzing exist- 
ing vocational assessment practices in the 
local district; 

3. Collaborative participation in the develop- 
ment and implementation of a plan for im- 
proving assessment services for students 
with handicaps; and 

4. Knowledge of vocational education 
programs and special services present in 
the school district. 
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Introduction 

This module consists of two separate but related parts. Part I provides 
guidelines for you and your team members to use in setting up a cur- 
riculum-based vocational assessment process. Specific steps to follow in 
determining assessment practices and needs in your district are provided. 
An important outcome of this team activity will be a plan of action for ex- 
panding and improving assessment services for students with handicaps 
in your vocational education programs. This plan of action will also serve 
as a basis for identifying particular modules in the rest of the training sys- 
tem that address specific needs in your district. 

Part II of this module is focused on information and procedures for en- 
suring that students with handicaps and their parents are informed and 
knowledge "hie about the programs available through vocational education 
in your d .s . ict. As a result of completing this section, you will become more 
knowledgeable about this relatively new service delivery requirement and 
about the full range of vocational education and special service options 
present in your district. As a team member in estabhshing a CBVA process, 
this information will be very helpful to you in determining how to integrate 
different assessment activities across all types of vocational programs in 
your district. 
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Part I: Teaming to Implement a CBVA Process 
Introduction 



Part I of this module is concerned with the process of establishing a new 
educational process in your school or district. It assumes that there are like- 
ly to be one of two scenarios in your school or school district: 

1. No current well-defined vocational assessment process 
is in place in your district, and there is some level of ad- 
ministrative and support services commitment to estab- 
lish a CBVA approach; or 

2. A more traditional vocational evaluation approach to 
vocational assessment is in place and there is some level 
of administrative and support services commitment to 
change to a CBVA approach. 

In either case, it should be evident that implementing a CBVA system 
is going to require a change in the way services are delivered to students 
with handicaps in your district. This is because CBVA is a comprehensiv e 
P rocess ' not just a one-time testing program. Since it is a process, occurring 
throughout a student's vocational education experience, CBVA requires 
support and input from multiple staff persons in the district, including ad- 
ministrators, counselors, teachers, and support services staff. Consequent- 
ly, moving to a CBVA model will call upon some individuals to change the 
way they teach or provide support services, or even the way they think 
about teaching and supporting students. This requires group dynamics; it 
requires a systems change approach to working with groups. That is how 
Part I of this module is designed to help you. 

The difficulty in changing a school system toward a CBVA process is 
likely to vary greatly depending upon a variety of factors including ad- 
ministrative and teacher support, availability of funds, staff expertise, and 
commitment to alternative models of assessment. Based on the field-test- 
ing of this training program in several diverse settings, however, there ap- 
pear to be several fundamental requirements that must be met in order for 
a district to begin to adopt a CBVA approach. First, someone must take 
ownership of the process in the district. Without a firm advocate for this 
approach, its future is not likely to be great. It is assumed that someone ex- 
ists in your district who fulfills this advocacy role, most likely a support 
services staff person or a district trainer. 

Second, this advocate murt be familiar with the district's ongoing as- 
sessment processes in vocational education, and with the vocational cur- 
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riculum options in the district. Again, it is assumed that this support ser 
vices staff person is familiar with the vocational curricula, or is in a posi- 
• tion to become familiar with it as you begin to implement CBVA. 

Finally, the district administration must have made a commitment to 
expending resources and staff time to an assessment process. CBVA, like 
more traditional vocational evaluation processes, requires resources and 
personnel. It requires them in a different configuration, but it requires 
them nonetheless. The district trainer and/or CBVA system advocate in 
your school district will need to devote a substantial amount of time in 
preparing the various personnel in the district for the elements of the CBVA 
process contained in the subsequent training modules in this series. He or 
she will likely serve as the group leader, initiate meetings, prepare agen- 
das, secure district commitments, arrange for incentives for staff to par- 
ticipate in the CBVA implementation process, and follow through on local 
planning. 

Guidelines for Implem enting Change 

Fullan (1982, pp. 178-179) discusses six guidelines to follow to imple- 
ment change in your school. It is important for you to be aware of these 
guidelines in order to assist your district trainer/advocate in following them 
on through the change process. These guidelines have been adapted to a 
CBVA context and are discussed briefly below: 

1. Choose a vocational classroom or group of classrooms 
which have personnel who are enthusiastic about 
change, and who see the need for it. Do not expect 
wholesale adoption of change by all vocational program 
areas. Start small with one or a few programs and use 
their successes as a model for the remainder of the facul- 
ty. 

2. Develop the capacity of this nucleus of staff to lead the 
change. Ensure that adequate training exists to help 
staff understand the dynamics of CBVA, and that peer 
support systems are developed to encourage and main- 
tain the use of CBVA through its most difficult begin- 
ning periods. 

3. Provide the necessary resources to support the adoption 
of change to a CBVA system. Incentives must be 
provided, along with a clear understanding that "this 
time, change is for real." Without such support and un- 
derstanding, instructors and support personnel may as- 
sume that this is just another fad, and enthusiasm may 
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fade quickly after the initial intensity of the training 
period is over. 

4. Develop a clear procedure for implementing the change. 
This requires group planning and a detailed documen- 
tation of activities, timelines, outcomes, and personnel 
responsible. A series of steps to develop such a proce- 
dure follows this introduction to Part I. 

5. Recognize that the implementation of a plan is, in itself, 
a difficult process. Problems in changing district voca- 
tional assessment policies and practices do not stop with 
the development of a plan to implement CBVA. Con- 
tinued support by administration and CBVA system ad- 
vocates to vocational teachers and support personnel 
must be maintained in order to sustain the momentum 
of the CBVA implementation. 

6. Monitor the process closely after initial successes with 
implementing the plan. Traditional practices in school 
districts are tremendously resilient and resistant to 
change even when stafif recognize the merits of chang- 
ing those practices. The most effective technique for 
monitoring changed practices is to develop and main- 
tain cooperative groups of teachers and support staff 
whose purpose is to systematically and continually com- 
municate the effects of changed practices with other dis- 
trict personnel, both horizontally (across teachers) and 
vertically (to administration and parents). 

Strategies 

Brody (1982, pp. 208-212) provides a detailed checklist of questions to 
consider as groups of staff persons in a school decide to engage in changing 
an educational process in their system. In addition, Kingsbury (1988), 
functioning in the role as a district trainer, provided a list of steps to follow 
as he worked with a field- test site in the development of these CBVA train- 
ing materials. These two sources of information have been collapsed 
together and displayed in a series of steps for you and your team members 
to follow in developing your CBVA process. In addition, you may want to 
consult Ianacone and Leconte (1986), and Rosenfield and Rubinson (1985) 
for other considerations in the systems change process of introducing cur- 
riculum-based assessment in your school. 
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Step 1: Clarify Perceptions of Need for Vocational Assessment 

It is important at the outset to gain an understanding of yo ir 
administrator's perceptions of the need for the school's vocational assess- 
ment, as well as the perceptions of teachers and support staff of those as- 
sessment responsibilities. This will involve clarifying assessment 
terminology, gaining different staff perspectives on the nature of ongoing 
vocational assessment activities, and the scope of those activities. The cut- 
come of Step 1 should be a mission statement identifying the school's 
responsibilities relative to vocational assessment of students with hand- 
icaps. The following procedures are recommended to achieve this outcome: 

A. Interview a group of individuals including ad- 
ministrators, counselors, key vocational teachers, and 
support staff to determine the school's vocational as- 
sessment mission; 

B. As a group, develop a mission statement if one does not 
exist; 

C. Circulate the mission statement (either the one your 
group has developed or an existing one ) to verify its com- 
prehensiveness and accuracy. 

Step 2: Develop a Problem Statement 

Our experience in field-testing these training modules consistently con- 
firmed two things: 

1. Vocational teachers, administrators, and support staff 
believed that their vocational assessment practices 
needed upgrading; and 

2. Once introduced to the concept of CBVA, these in- 
dividuals believed that it would help to solve their voca- 
tional assessment problem. 

However, we also found that different individuals in the same district 
or school perceived problems with their assessment services differently, 
with varying perspectives on the same problem. This was due to the fact 
that different professionals approached the mission and need for vocation- 
al assessment from diverse positions in the educational process, and had 
multiple expectations of the utility of vocational assessment. For example, 
a vocational administrator may view vocational assessment of students 
with handicaps as strictly denning those students who are eligible for sup- 
port services; a guidance counselor may only be interested in identifying 
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the most appropriate vocational program placement; a special needs staff 
member may be looking for vocational assessment information to generate 
goals and objectives on the student's IEP; and a vocational teacher may 
look to vocational assessment to help monitor the progress of a student with 
a handicap in his/her program. 

Thus, the nature of a district's vocational assessment problem needs 
defining, including defining terms, establishing perspectives to the 
problem, and delineating the expectations that vocational assessment is 
designed to fulfill across different personnel. The outcome of this problem 
definition process should be a statement that captures the nature and scope 
of the vocational assessment problems from the multiple perspectives of 
the individuals involved in the CBVA implementation process. The follow- 
ing procedures are recommended to achieve this outcome: 

A. Interview key personnel relative to their views about 
the strengths and weaknesses of the present vocation- 
al assessment process. Then ask them to define what 
vocational assessment should produce for each of them; 

B . Develop a problem statement that attempts to capture 
as wide a perspective as possible of the different 
individuals' ideas about vocational assessment needs. 
An example might be: "Smith High School lacks a voca- 
tional assessment process that adequately identifies (1) 
who the special needs students are before they begin to 
have difficulty in their vocational program; (2 ) what cur- 
ricular or instructional modifications should be in place 
for them; and (3) how well they are progressing through 
their program;" and 

C. Circulate the problem statement to key individuals in 
your school for input, clarification, and revision. 

Step 3: Develop Needs Statements for Different Personnel 

As a variety of professionals provide input into the nature of the voca- 
tional assessment problem, a number of differing needs will emerge. Those 
needs should be collected and organized in order to provide a benchmark 
against which a CBVA process can be evaluated. As with Steps 1 and 2 
above, this step can be accomplished individually by yourself, or collective- 
ly through the use of a CBVA team of district individuals who are all par- 
ticipating in one or more of the training modules in this series. The outcome 
of this step would be a prioritized list of needs statements that should be 
accommodated by a vocational assessment system. 
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Step 4: Learn the Characteristics and Qualities ofCBVA 

At this point, it will be necessary to become familiar with how CBVA 
can address the general problem identified in Step 2, and the specific as- 
sessment needs delineated as an outcome of Step 3. It should be the respon- 
sibility of the CBVA advocate or district trainer to provide you with the 
materials and presentation time you will need to achieve that familiarity. 
Generally we anticipate this familiarity could be accomplished through a 
series of presentations and readings from the Trainer's Manual, which ac- 
companies Modules 1-7 of this series, and by reading Module 2\ "Under- 
standing Curriculum-Based Vocational Assessment: Purposes and 
Characteristics." The outcome of this step would be a clear idea by you and 
others involved in this training process of what CBVA is, and how it ad- 
dresses vocational assessment problems and needs in your school. The fol- 
lowing procedures are recommended to achieve this outcome: 

A. Attend all presentations on CBVA by your district 
trainer; 

B . Read selected readings and materials from the Trainer's 
Manual; 

C. Read Module 2 of this series 

D. Prepare a brief paper on + a nghowCBV «an general- 
ly address the problem statement developed through 
Step 2, and the specific needs statements identified in 
Step 3. 

Step 5: .Develop Goals and Objectives for Implementing CBVA 

At this time, you and other CBVA team members should have a fairly 
clear idea of how CBVA can address problems and needs in your school, 
and some notions of how it might be implemented. At the same time, you 
should be starting to read and complete assignments from other modules 
in the series, further solidifying your understanding of the elements of the 
system. It should then be possible for your team to develop a goal statement 
for implementing CBVA in your district, and a number of action-oriented 
objectives to address that goal statement. The outcome of this step will be 
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the goal(s) and objectives you have developed. Procedures to follow to 
achieve this outcome would include the following: 

A. Brainstorm, as a group, the goal(s) and objectives; 

B. Circulate these goal(s) and objectives with other key ad- 
ministrators and faculty members for additional input; 

C. Revise your goal(s) and objectives as needed. 

Step 6: Develop Action Plans for Each Objective 

Each objective identified in Step 5 above should have an action plan as- 
sociated with it, outlining very specific activities, anticipated beginning and 
completion dates, products of those activities, and personnel responsible 
for completing them. Example Document #1 displays a generic action plan 
foncat which can be used to complete this step. It is critical that the in- 
dividuals who are assigned responsibilities on these action pians be in- 
volved in their development, either initially as the plans are drafted, or 
through a review and refinement process. It is also recommended that 
structured, group planning techniques such as the Nominal Group Techni- 
que (Delbecq, Van de Ven, & Gustafson, 1975) or other group techniques 
(Brody, 1982) be used. By using a structured, group planning process, it 
will be assured that all members of the CBVA team have an opportunity to 
share their opinions about how the action plans should emerge. The out- 
come of this step would then be a finalized sequence of action plans address- 
ing each CBVA objective. 

Step 7: Identify Constraints and Incentives Needed to Implement Action 
Plans 

The systems change literature is extremely clear in its discussion of the 
need to identify constraints to implementing change, and incentives needed 
to overcome those constraints, Many researchers (c.f. Wade, 1985-86; 
Showers, Joyce & Bennett, 1987; Rosenfield & Rubinson, 198i,, Fullan & 
Pomfret, 1977; House, 1975) have concurred that such factors as ad- 
ministrative support, cooperative peer support, tangible incentives, and 
sponsorship by outside agencies/organizations, greatly increase the 
likelihood of a new educational process being adopted and maintained in a 
school. By identifying constraints which can undermine achievement of ac- 
tion plans, and incentives which address those constraints, the CBVA team 
may avert early failures in implementing these plans. 
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Step 8: Field-Test Action Plans 

Although district-wide implementation of the CBVA process is most 
desirable (Ianacone & Leconte, 1986), long-term success may be more easi- 
ly realized through starting with small-scale demonstrations of CBVA ac- 
tivity, and letting its reputation facilitate greater adoption across the school 
or district (Rosenfield & Rubinson, 1985). Thus, as mentioned in the intro- 
duction to Part I of this module, start with a single teacher or small nucleus 
of teachers, and pilot-test the process with them. Action plans and proce- 
dural processes can then be developed and ^efined to streamline the adop- 
tion across a wider target group. 

Step 9 : Implement Action Plans and Monitor Effects 

This last step involves the full-scale implementation of action plans 
after they have been streamlined through field-testing. It will be necessary 
to keep a close watch on the implementation of the plans, maintaining 
flexibility in case goals and objectives change, personnel change, or ac- 
tivities fall behind schedule. Probably most important in this step will be 
the realization on the part of the CBVA team that change can be very slow, 
that the need for constant adaptations may arise and should be encouraged, 
and that successes that are less in scale than originally planned are still 
successes, not failures. 
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Part II: Communicating Program Options 
in Vocational Education 1 

Introduction 

The requirement that students with handicaps and their parents be in- 
formed of the opportunities available through vocational education first ap- 
peared in the Carl D. Perkins Vocational Education Act of 1984(PL 98-524). 
This assurance is fairly specific, as noted in an excerpt from the law itself: 

"Each local educational agency shall provide information to hand- 
icapped and disadvantaged students and parents of such students 
concerning the opportunities available in vocational education at 
least one year before the students enter the grade level in which 
vocational education programs are first generally available in the 
state, but in no event later than the beginning of the ninth grade, 
together with the requirements for eligibility for enrollment in 
such /ocational education programs. (Section 204.)" 

Since this requirement is relatively recent, and parents, students, and 
local CBVA team members alike need to have this information to effective- 
ly communicate about instructional program options, this part of Module 
1 focuses on three important questions: 

1. What types of information about vocational education 
programs need to be communicated: 

2. How can this information best be disseminated to meet 
the letter and spirit of the law? 

3. Who, in addition to students and parents, would benefit 
from information about vocational education programs? 

Strategies relative to each question are presented in the next section. 



The original field-test version of this section was prepared by Michelle Sarkees 
and Nick Elksnin. Special thanks are extended to both for their important 
contribution to this training system, 
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Strategies 

Types of Program Information Needed 

Sarkees (1986) has described the types of vocational program informa- 
tion that are necessary for vocational decision-making. These include: 

1. A brief description of specific vocational education and 
training programs available at the local level; 

2. Eligibility criteria for specific programs; 

3. Admissions procedures for specific programs; 

4. Basic skills, knowledge and employability skills that 
employers expect from entry level workers and how they 
are a part of vocational education curricula; 

5. Job opportunities available with completion of a voca- 
tional program or a portion of it (multiple exit points); 

6. An approximate amount of time the students will typi- 
cally spend in vocational education coursework; 

7. Examples of methods available to vocational teachers 
to teach entry level or transferable skills to students 
(e.g., cooperative education, supervised occupational ex- 
periences, simulated work experiences, participation in 
vocational student organization activities); 

8. Special support services available to students with 
handicaps enrolled in vocational programs; and 

9. A name address and phone number of a local contact 
person. 

This information list should be considered minimal, and could be sup- 
plemented with other information as necessary. 

Activities for Communicating Program Information 

An extensive list of methods for disseminating vocational program in- 
formation is presented in Example Document #2 . In reviewing this list, con- 
sider the use of a variety of traditional and nontraditional methods to 
ensure that the information is indeed being received and understood by the 
intended audiences. This consideration is especially important for in- 
dividuals who have difficulty with print (e.g., limited English proficient 
parents, parents with low reading levels, parents who do not usually at- 
tend formal school functions or counselor/teacher/IEP meetings). 
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An excellent avenue for communicating vocational education program 
options to students with handicaps, and one that many schools are using 
is through the Individualized Education Program (IEP) process. The an- 
nual development, review and revision of a student's IEP is a process that 
is mandated by federal law (PL 94-142) and currently is in place for all stu- 
dents with handicaps. 

Some potential benefits for utilizing the IEP process are that it: 

1. Eliminates the need for creating a new process and 
potential duplication of effort; 

2. Insures the inclusion of vocational education on the IEP; 

3. Eliminates potential "dumping" of special students into 
inappropriate vocational programs because each 
student's program is developed utilizing information 
concerning: vocational program interest, availability of 
support services, potential for program modification, 
current levels of educational performance and in- 
dividual needs of the student; 

4. Provides a yearly mechanism for review; 

5. Provides a forum for informed decision making; and 

6. Provides due process procedures for the student, parent 
and school district. 

The IEP process is described extensively in the literature (c.f M Turnbull, 
Strickland, & Brantley, 1982; Sarkees & Scott, 1985; Turnbull & Schultz| 
1985). As the IEP is a process that vocational educators have not been 
primarily involved with, and local procedures vary considerably, 
familiarization with the IEP process at the local school level is recom- 
mended. 



Targeted Audiences 

Thus far, the information presented has been directed to two primary 
audiences: 

1. Students with handicaps; and 

2. Their parents. 

However, during the field-testing of this training program, the par- 
ticipants emphasized the value of learning more about the existing voca- 
tional programs and special services in their respective districts. Discussion 
with vocational teachers, special educators, guidance counselors and ad- 
ministrators from the participating schools revealed that by having an up- 



14 



Modult 1 

Curriculum-Based Vocational Assessment 



to-date and common understanding of the various programs available -id 
their requirements, the participants were better prepared to work on CBVA 
activities as a team. Thus, it is recommended that vocational ad- 
ministrators plan to communicate vocational program information to 
teachers and staff within their own facility as well as to potential students 
and their parents. One excellent technique to accomplish this would be to 
highlight a vocational program at each weekly or bi-weekly staff meeting, 
and include a short, standardized presentation by the instructors) of their 
curricular goals, instructional processes, and equipment and laboratory 
facility. 

Summary 

This module has been concerned with establishing a CBVA process in 
your district. In order to do this, information and strategies have been 
provided to: 

A. Assist you with the difficult process of starting an in- 
novative process in a school district; and 

B. To find out more about your own programs so you can 
communicate such information to students and parents, 
and understand more fully the range of programs which 
would be included in a CBVA process. 

The final segment of information you will need to get started is a 
thorough understanding of CBVA as a concept, and what it encompasses. 
Module 2 is designed to help fulfill that need. 
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Here are some methods for providing information about vocational 
education opportunities to students with handicaps and their parents 1 : 

Activity Description 



Brochures, Flyers 
and Leaflets 



Career Day 



Newsletter 



Brochures, flyers and leaflets describing the 
vocational programs available in the local dis- 
trict, Information should include: 

1. Basic program information; 

2. Eigibility requirements; 

3. Admission policy; 

4. Job opportunities; 

5. Contact information (name, address, 
phone); and 

6. School policy statement on nondiscrimina- 
tion. 

This activity can be set up in a gymnasium, 
auditorium or in each individual vocational 
classroom laboratory. Employers and advisory 
committee members can be available to talk 
about employment opportunities in the com- 
munity. Vocational teachers talk to students 
and parents about program activities, multiple 
exit points and entry level competencies that 
students can be prepared for in the vocational 
program. If the Career Day activity is based in 
the vocational classroom/laboratory, teachers 
and students can demonstrate equipment, tools 
and procedures that represent the program. 
Programs available for further development of 
skills at the post-secondary level should be in- 
troduced. 

A school newsletter can be produced peri- 
odically throughout the year to provide up- 
dated information on the activities being 
conducted in each vocational program (e.g., 



Adapted from Sarkees f M.D. (1986, December). Commafuca^'ng Vocational Op- 
tions to Special Needs Students and Their Parents. Paper presented at the 
Special Needs Divisi< n Symposium on Comprehensive Programming for 
Special Needs Students, American Vocational Association Convention, 
Dallas, TX. 
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Activity 



Description 



Parent Letter 



Video Presentation 



TV and Radio Spots 



program highlights, vocational student or- 
ganization activities/events, contact person for 
specific program information). 

A letter can be sent to the parents/guardians -tf 
every identified handicapped student. The 
ter should contain, at a minimum: 

1. Description of vocational educat ; ...avail- 
able local programs, and pos jie job op- 
portunities; 

2. Admissions requirement; 

3 . Name, address and phone nuro her of con- 
tact person; and 

4. Nondiscrimination statement. 

If there are limited English-proficient popula- 
tions in the community the letter should be 
written in the native language spoken in the 
home. Many districts are sending letters to 
parents of every eighth grade student. Some 
districts are asking the parent/guardian to sign 
the letter and return it to the school. 

Video "spots" about vocational education oppor- 
tunities can be developed for use on local tele- 
vision stations. These short segments can 
present information on the goals of vocational 
education, local secondary programs, employ- 
ment opportunities available in the com- 
munity, post-secondary opportunities, a local 
contact person and phone number. These spots 
can be produced in the native language spoken 
in the community to meet the needs of limited 
English proficient populations. They can also 
be used for local community groups, PTA meet- 
ings, etc. 

Spot announcements and news items can be 
developed for use on local radio and television 
stations. Information could focus on the goals 
of vocational programs, admission require- 
ments, local programs available (secondary and 
post-secondary) employment opportunities, 
local contact name, address and phone number! 
These segments can be done in the native lan- 
guages spoken in the homes in the community. 



SB 



22 



Module f 

Curriculum-Based Vocational Assessment 



Activity Description 



In addition to short "spots," panel discussions, 
debates, demonstrations, guest appearances 
and interviews can be prepared. A nondis- 
crimination statement should be included in 
every presentation. 

Exploratory Students at the middle school level can become 

Experiences involved in exploratory experiences in existing 

vocational programs. After participating in a 
number of programs, student responses and 
preferences can be recorded. A copy of this in- 
formation, along with programs in which stu- 
dents register a strong interest, can be sent 
home to the parent/guardian with program 
descriptions and admission criteria. Another 
copy should be put in the student file in the 
guidance office for use in the student/parent 
conference to plan the student's high school 
program. 

Student Assembly An assembly can be arranged to include all 

eighth grade students. Information about local 
vocational programs can be presented. Voca- 
tional teachers and employers can be in- 
strumental in providing specific information on 
program opportunities, sequence of courses for 
graduation, skills to be developed in the 
program, vocational student organization ac- 
tivities, and employment opportunities. 
Brochures, leaflets, student handbook?, memos 
or letters can be sent home to parents/guar- 
dians after the assembly. Parents can also be 
invited to be a participant at the assembly. 

Counselor Meeting Many eighth grade students have individual 

conferences with the counselor and the 
parent/guardian. During these sessions, the 
direction the student will take in high school 
and beyond is usually discussed and mapped 
out. Results of exploratory experiences in voca- 
tional programs, interest inventories and other 
relatedinformation should be presented during 
the conference. Parents could be invited to visit 
prospective programs and teachers at this time. 
Brochures, slide presentations and video 
"spots'* can be used to provide further informa- 
tion after the conference. Counselors can also 
introduce parents to support personnel who 
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Activity 



Description 



Open House 



Newspaper Releases 



Presentations by 
Vocational Teachers 



Displays at Places 
of Public Gatherings 



can describe services that are available to the 
student. 

This can be either school-wide or limited to the 
vocational programs available in the district. 
Notification should be made to all parents in 
the native language spoken in the home, invit- 
ing them to attend the open house. At this time, 
program information can be shared and actual 
demonstrations can be conducted to show re- 
lated skills taught in the program. Employers 
and advisory committee members can also be a 
part of this event. Support personnel can be 
present to describe the services available to 
help students succeed in the program. 

News releases and articles can be used to 
describe vocational program opportunities to 
parents in the community. Specific information 
can be provided about program descriptions, 
eligibility requirements, admission criteria and 
procedures and the name and number of a local 
contact. In addition, information can be in- 
cluded about the progress of vocational student 
organizations on school and communication 
projects, news from program graduates, recog- 
nition of specific student achievements, results 
of fund raising drives, and new courses or 
program authorizations. 

Vocational teachers can visit middle school 
classes individually or in small teams. They can 
provide students with information about 
program opportunities, vocational student or- 
ganization opportunities, potential employ- 
ment possibilities, courses to take at the 
secondary level and post-secondary options. 
Brochures, leaflets, pamphlets or other infor- 
mation can be passed out for students to take 
home. Vocational student organization mem- 
bers can accompany the vocational teachers to 
provide another perspective. 

Displays at malls, community centers, fairs, 
cultural events, church events, etc. can provide 
information to community members about 
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Activity 



Description 



"Sidewalk Cafes" 



Presentations to 
Community Groups 



Student Handbook 



Telephone Calls 
to Parents 



vocational education opportunities on the 
secondary and post-secondary level. This infor- 
mation should be printed in the language 
spoken in the homes of the community mem ■ 
bers. 

This activity is usually conducted after school 
hours (evenings, week-ends) and involves small 
gatherings that meet informally "over coffee* to 
discuss information about vocational education 
opportunities. This could occur at a student's 
home with several neighbors, at a Neighbor- 
hood Center, or after a community meeting. 
Vocational teachers, counselors, special educa- 
tion coordinators and vocational student or- 
ganization members can all volunteer to take 
part in this activity. Brochures could be dis- 
seminated with a contact name and number for 
further information. 

Vocational teachers, special education 
teachers, counselors, administrators or voca- 
tional student organization members can share 
information about vocational program oppor- 
tunities to local community groups. Slide tape 
and video presentations can be incorporated 
into these activities. General questions can be 
addressed. Brochures, pamphlets and leaflets 
can be disseminated. An interpreter should be 
available to work with limited English-profi- 
cient participants. 

A student handbook is usually made available 
to every eighth grader to provide information 
about options available at the secondary level. 
One section of the handbook should describe 
vocational education as an option for students 
at the secondary level. At a minimum, this sec- 
tion should include a description of what voca- 
tional education is, as well as a brief narrative 
for each program available at the l<~'il level, 
courses to be taken in high school, admissions 
procedures, eligibility criteria, and possible 
employment opportunities. 

Special education teachers or counselors can 
make personal phone calls to the parent/guar- 
dian of identified special needs students in the 
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Activity Description 



eighth grade to discuss the option of vocational 
education. Specific information should be 
provided at this time. An invitation can be 
made for parents to visit the school, teachers, 
and vocational programs. Interpreters should 
be available to furnis h information for limited 
English-proficient parents. 
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Module 2 

Understanding Curriculum-Based 
Vocational Assessment: 
Purposes and Characteristics 

GOAL: Upon completion of this module, you will be 

knowledgeable of the central features of a 
CBVA process and how this training program 
relates to the CBVA process. 

COMPETENCIES: In order to understand the purposes and 

characteristics of CBVA, you should 
demonstrate: 

1. Knowledge of the central features of a 
CBVA process; 

2. Understanding of why these CBVA fea- 
tures are important in vocational educa- 
tion and training ettingsjand 

3. Knowledge of the relationship between a 
CBVA process and the content of this 
training program. 
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Introduction 

Module 1 of this training system provided you with information about 
how to work collaboratively as a team in effecting change in the way your 
district conducts vocational assessments for students with handicaps. It 
also provided you with information about communicating vocational educa- 
tion and training opportunities in your district to students and their 
parents. 

This module is also designed to fulfill two purposes. Part I is titled 
"What is CBVA?" and gives you detailed information about the characteris- 
tics and features of curriculum-based voca onal assessment. By learning 
about what CBVA is as a total assessment concept, you will be able to un- 
derstand how it responds uniquely to each of the assessment purposes that 
form the backbone of each of the subsequent modules in this training sys- 
tem. 

Part II of this module is titled "The CBVA Training System." This sec- 
tion introduces each of the modules in this training system, and explains 
how they function as an integrated system. When you have completed this 
part and participated in discussions of this module with your team and the 
district trainer, you should be well prepared to begin completing the 
remaining modules in this training system. 
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Parti; What is CBVA? 1 

Vocational educators at state and local agency levels have long recog- 
nized the importance of providing individualized assessment and program 
planning services to students with handicaps. Now, however, with the tar- 
geting of assessment services for special needs learners in recent federal 
vocational education legislation, many program managers are in the midst 
of making fiscal and programmatic decisions about the type of assessment 
services that need to be in place in local vocational education settings. 

An often seen tendency is for the program manager to hastily invest 
available funds into one of the many commercially produced vocational as- 
sessment systems currently on the market. Given the attractiveness of 
these systems, the apparent completeness of the assessment package, and 
the short time period in which it usually takes to start up an assessment 
center, the investment seems quite reasonable at the time. Unfortunately, 
however, not long after the particular system is installed, many program 
managers have discovered that the types of assessment services needed by 
students with handicaps in - -itional education far exceed those provided 
through the commercial assessment system route. At this point, alterna- 
tive approaches may be explored, but usually with fewer dollars than the 
amount originally invested in the commercial program. 

The type of assessment services presented in this document and the 
companion series of training modules is commonly referred to as Cur- 
riculum-Based Vocati o nal Assessment (CBVA). This approach may well 
provide the assessment options that program managers find lacking in 
more traditional approaches. The CBVA approach, which is characterized 
by the use of performance-based assessment procedures developed and 
used by local vocational and special services personnel, is not a new idea in 
vocational education circles. Its use was advocated in the 1970s (Phelps & 
Wentling, 1977; Albright, Fabac, & Evans, 1978; Hartley, Otazo, & Cline, 
1979; Kok & Parrish, 1979). However, further developmental work in CBVA 
has taken place in recent years in order to assist local school districts in 
responding to the comprehensive assessment and programming provisions 
contained in the federal statutes governing vocational and special educa- 



An earlier version of Part I of this module appeared in: Albright, L. & Cobb, R. 
B. (1988). "Curriculum-based vocational assessment: A concept whose 
time has come." Journal for Vocational Special Needs Education, 10(2), 
13-16. ~~ 
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tion (Sitlington, 1981; Cobb, 1983; Albright & Hux, 1984; Cobb & Larkin, 
1985; Ianacone & Leconte,1986; Minnesota Assessment Guidebook, 1986; 
Stodden, Ianacone, Boone, & Bisconer, 1987). 

Basically, CB VA is a continuous process used to answer questions about 
the instruction and special service needs of individual students as they 
enter into and progress through specific vocational education programs. 
Several distinguishing features of the CBVA process are: 

1 . It responds to the information needs of personnel during 
the selection and planning stages of a student's voca- 
tional program, as the student progresses through the 
program, and during the transition of the student from 
school to productive employment in the community 
and/or post-secondary education; 

2. Assessment activity is tied directly to, and is an integral 
part of the student's vocational education curriculum; 

3. Personnel responsible for providing vocational instruc- 
tion and special services to the student are the in- 
dividuals who collaborate in conducting CBVA 
activities and who utilize the results to ensure success- 
ful student performance in vocational education and 
employment settings; 

4. It involves the use of informal and direct procedures for 
determining student achievement in local vocational 
education programs; and 

5. It is adaptable to community-based training environ- 
ments at both the secondary and post-secondary level 
for individuals with a full range of handicaps. 

Further discussion of each of these and other characteristics of the 
CBVA process follows. 



Continuous Process 



Often when the term vocational assessment is mentioned, one thinks 
of a series of specialized activities that take place prior to the student's 
entry into a vocational education program. While this perception is partial- 
ly correct, we know that the assessment of student interest anrl achieve- 
ment in vocational education is actually an ongoing matter. Such 
information is needed during three different stages of a student's program: 

1. When the program placement decision is in process; 

2. During the student's participation in the program; and 

3. As the student exits the program. 
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Within each of these stages, a different set of questions about student 
progress and special service needs comes forth. An example of these stages 
and the questions that emerge in each are shown in Table 1. These ques- 
tions, and the answers to them, are needed by the student's vocational in- 
structor and special service personnel. CBVA recognizes the ongoing need 
for information about the student in vocational education and the neces- 
sity for coordinated information gathering and d jcision-making activity be- 
tween vocational and special services personnel. 

Focused Asses sment Activity 

Another important feature of CBVA is a concentrated focus on the stu- 
dent within a particular vocational program. Assessments are, therefore, 
centered around the vocational program setting. The notion here is that the 
unique service needs of the students are best determined by assessing the 
student directly in the setting in which he/she is expected to perform. For 
example, during the student placement stage, a situational assessment 
such as a tryout in the vocational program setting could be used. Or, in 
terms of monitoring student progress, a series of direct observations of stu- 
dent performance in the program environment may be used. By keeping 
the assessment function focused on the program context, the results of such 
activity will be more closely linked to the instructional requirements of the 
particular program. 

Use of Informal and Direct Proced ures 

Vocational and special services personnel regularly assess student in- 
terest and achievement through a variety of techniques like direct obser- 
vation, knowledge and performance testing, and interviewing. The intent 
of CBVA is to build upon these naturally occurring activities in a manner 
that eventually develops into a unified series of assessments for determin- 
ing student performance in vocational education. Here again, the notion of 
integrating assessment activity with the ongoing instructional program is 
reinforced. 

Personnel 

Another distinguishing characteristic of CBVA is that the personnel 
directly responsible for providing vocational instruction and special ser- 
vices to the student are the ones who ask the questions about the student's 
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Table 1: Program Stages and Sample Assessment Questions 



Stage 



Assessment during 
program placement 
and planning 



II. 



Assessment during 
participation in voca- 
tional program 



III. 



Assessment during 

program 

exiting 



Definition 



Activities that occur prior to 
and during first few weeks 
of student participation in a 
vocational program 



Activities that take place as 
the student progresses in a 
vocational education 
program 



Activities that occur near the 
end of student *s program 
and following completion 



Purpose 



Assessment 
Questions 



• Program selection 

• Program placement 

• Program planning 



• Monitor student progress 

• Determine appropriate- 
ness of program and ser- 
vice delivery plan 

• Evaluate success of 
student* s program 

• Plan for future service 
needs of student 



Which vocational program is 
most appropriate for the student? 
What arc the special service 
needs of the student in this par- 
ticular program? 
What will be the criteria used to 
detcmiine student success? 

How is Hie student performing in 
the vocational setting? 
What changes arc needed in 
student's program? 



What arc the special services 
needed to help the student transi- 
tion into employment and/or post- 
secondary education? 
Which adult service agencies 
need to be linked up to the stu- 
dent? 

How will student adjustment be 
monitored? 
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program and who obtain the information necessary to make program 
decisions. The direct involvement of these personnel will ensure that the 
information derived from assessment activity is closely linked to the re- 
quirements of the local program and that the important concerns of instruc- 
tional and special services personnel are being answered in a direct, 
efficient and collaborative manner. 

Cost Considerations 

Since CBVA utilizes existing vocational and special services personnel 
in the district, the start-up and operational costs of a local CBVA process 
are largely in personnel training. In order to develop a comprehensive and 
valid local CBVA process, the various personnel involved in assessment and 
decision-making activities will need to be identified and trained. For some 
individuals the requirements of a local CBVA effort may involve minimal 
training time and change in their daily routine. For others, however, sub- 
stantial training may be necessary. To illustrate, a vocational teacher may 
already be using many of the informal assessment techniques necessary to 
complement the CBVA process, and, therefore, only minor adaptations may 
be needed. However, for the individual who has recently assumed the posi- 
tion of coordinator of vocational special needs services, a substantial 
amount of training time may be needed to fully understand and implement 
the CBVA process. Even so. the need for training would probably only 
amount to the equivalent of one or two graduate level courses in assess- 
ment. 

Another important consideration is in the shifting of personnel roles to 
accommodate the CBVA process. Some individuals may, for example, be 
collecting important assessment information, but it could be collected in- 
frequently and seldom communicated to the various personnel responsible 
for monitoring the student's special services program. While training on 
collecting and reporting assessment information in a coordinated manner 
should improve this situation, other adjustments, like restructuring the 
workload to inc.ude collaborative planning time, may be necessary. A first 
step in establishing the local CBVA process will be to u ermine present 
personnel roles and practices in relation to those required for operating the 
CBVA process. 

Finally, since assessments are conducted within existing vocational 
programs, extra funds are not necessary for special facilities and equip- 
ment. Furthermore, the use of informal, locally developed assessment pro- 
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cedures eliminates the need for large expenditures on standardized testing 
instruments ar equipment. 

Adaptability to Community-Based Training Envi ronments 

A final characteristic of the CBVA approach to vocational assessment 
is its adaptability for use outside the scope of traditional vocational educa- 
tion programming. The assessment processes and techniques inherent in 
CBVA can be used equally well in community-based training settings for 
two reasons: 

1. CBVA processes are not inhibited by laboratory and 
equipment constraints; and 

2. CBVA activities are focused around assessment pur- 
poses that are common to both vocational education and 
training environments. 

For example, special services personnel may decide in a staffing meet- 
ing that preparation for employment is an important goal for a particular 
student, but existing vocational education curricula are either too narrow 
in focus, or are not of interest to the student. Instead, a community-based 
placement and training program appears to be a better option. However, 
there still needs to be an assessment of which specific type of job the stu- 
dent desires, the nature and intensity of supportive services to be provided, 
monitoring of those services, and considerations of transferring support on- 
the-job to adult services agencies, if necessary. The same assessment tech- 
niques that fulfill these functions in vocational education laboratories will 
work in this community-based training environment as well. In fact, during 
the field-test phase of this CBVA training system, we found that CBVA 
techniques provided excellent assessments in community-based work ex- 
perience programs at the post-secondary level in Colorado, in personnel 
selection activities for an industrial firm in Minnesota, and for supported 
employment programs at the adult service level in Vermont. 

The Lnaracteristics of CBVA discussed above clearly show it to be an 
excellent option for districts which desire to implement assessment proces- 
ses that respond to the full range of information needs as a student progics- 
ses through his/her high school vocational education experiences. Most 
important, however, is that an assessment process must yield information 
in a valid, reliable, and efficient manner. Given these requirements, and 
the number of other excellent features of CBVA, we believe curriculum- 
based vocational assessment to be a process worth adopting in virtually 
every secondary vocational education program. 
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Part II; The CBVA Training System 

The modules in this training system are organized around a chronologi- 
cal sequence of service delivery components. Table 2 presents a schematic 
of these service delivery components and indicates how they interrelate. 

As can be seen in Table 2, you and your team members would begin with 
the task of establishing your district CBVA process and also becoming more 
familiar with the full range of vocational education programs offered in the 
district. Module 1 will assist you and your team in completing both of these 
activities. Either prior to or during these Module 1 activities, you are also 
becoming more aware of the common characteristics and features of a 
CBVA process and the CBVA training system, via Module 2. 

The information is Modules 3-6 represent the chronological sequence of 
student assessment activities that were suggested earlier in Table 1. More 
specifically, Module 3 material pertains to assessment activities for student 
placement in vocational education. Module 4 provides a set of guidelines to 
use in planning the student's vocational education program. The informa- 
tion in Module 5 offers a number of strategies to use in monitoring student 
progress in a specific vocational program. The material in Module 6 will be 
helpful to you and your team members in assisting the student in making 
a successful transition from the vocational education program to employ- 
ment and/or advanced training. Hence, Modules 3 and 4 relate to the first 
stage of assessment shown earlier in Table 1; that is,assessment activities 
which occur prior to and during student entry into vocational education. 
Module 5 focuses on those assessment activities that take place while the 
student participates in the program, which is the second stage of assess- 
ment displayed in Table 1. Module 6, the transitional assessment module, 
occurr, near the end of the student's program and following program com- 
pletion. This represents the third stage of assessment shown in Table 1. 

The last module, Module 7, provides you and your team members with 
a means for evaluating the effectiveness of the district's CBVA process. 
Specific steps to follow in organizing and conducting the evaluation are 
presented. 

Finally, Table 3 clearly shows the variety of personnel and assessment 
instruments involved in the development, use and evaluation of a CBVA 
process. Each of the subsequent modules in this training sequence will 
specifically address the manner in which personnel and instruments are to 
be used. 
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Table 2: CBVA Training Module Sequence 



MODULE 1 

Establishing a 
Curriculum-Based 
Vocational 
Assessment 
Process 



MODULE 2 

Understanding 

Curriculum-Based 

Vocational 

Assessment: 

Purposes and 

Characteristics 



MODULE 3 

Placing 
Students in 
Vocational 
Education 
Programs 



MODULE 4 

Planning a 

Student's 

Vocational 

Education 

Program 



MODULE 5 

Monitoring 

Student 

Progress 



MODULE 6 

Planning 

Transitional 

f ? "rvices 



MODULE 7 

Evaluating 
the CBVA 
Process 
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Table 3: CBVA Implementation Guide 



, F£W Key to Personne l 

VSNT s Vocational Special Needs Teacher 
SSE = Secondary Special Educator 
VI = Vocational Instructor 



Purpose 
For 

Assessment 



Module 1 

Parti: 

Establishing a 
CBVA Process 



Part II: 

Communicat- 
ing vocational 
education 
program infor- 
mation 

Module 3 

Placing stu- 
dents in voca- 
tional 
education 
program 

Module 4 

Planning a 
student's voca- 
tional educa- 
tion program 

Module 5 

Monitoring 

student 

progress 

Module 6 

Planning tran- 
sitional ser- 
vices 

Module 7 

Evaluating 
CBVA process 



Timing of 
Assessments 



Personnel 
Involved in 
Assessment 
Activities 



Ongoing 



Starts at least one 
year prior to when 
vocational education 
is routinely available 
in district 



Semester prior to 
when vocational 
programs begin 



During annual IEP 
review and first 
month of vocational 
class 



Throughout duration 
of vocational program 



During semester 
prior to graduation 



Ongoing — annually 



VSNT 
SSE 
VI 
*VA 

VSNT 
VC 
VI 
SSE 



VSNT 
VC 
SSE 
VI 



VJSNT 
SSE 
VI 



VSNT 
♦ VI 
SSE 
VC 



VSNT 
SSE 

yc 

VI 



*VSNT 

VA 
VI 



VC a Vocational Counselor 
VA s Vocational Administrator 
* Suggested Lead Person(s) 

Instruments/ 
Procedures Used In 
Assessment Activities 



CBVA Action Plan 



Review process to assure identifica- 
tion of all students with handicaps 
in district and their parents are in- 
formed of options 



Review of student records; inter- 
view with student and parents; as- 
sessments in vocational try-out 
program 



Direct observations of student per- 
formances; inventory of program en- 
vironment and curriculum 



Develop a monitoring system for in- 
dividual students; conduct process 
and product assessments of student 
performance* 



Review of student performance; in- 
terview with student and parent 



Annual review process to evaluate 
CBVA processess/activities 



Note: 



Module 2, Understanding Curriculum-Based Vocational Assessment, is omitted from this 
table; however, all CBVA team members will need to carefully review and discuss its con- 
tents, 
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Module 3 
Placing Students in 
Vocational Education Programs 

GOAL: The information in this module will assist you 

and your CBVA team members in developing a 
systematic process i determining appropriate 
vocational education placements for students 
with handicaps. 

COMPETENCIES: As a result of completing this module, you will 

demonstrate skills in the following assessment 
for placement areas: 

1. Working as a team member in student as* 
sessment and program placement func- 
tions; 

2. Determining student and parent interests 
in vocational education; 

3. Organizing and conducting an exploratory 
program for students; 

4. Conducting assessments of students in- 
vocational tryout activities; 

5. Analyzing vocational program exploration 
information for student placement 
decisions; and 

6. Utilizing assessment information in stu- 
dent placement decisions. 
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Introduction 

The decision to place a student with a handicap into a vocational educa- 
tion program involves several individuals. The student, for example, is ex- 
pected to make a vocational program choice based on her or his career 
interests. In order to make an informed decision, the student will need to 
know about the various training options available, t'le requirements for 
successful program completion, and the prospects lor subsequent employ- 
ment in the occupational area. Likewise, the preferences of the student's 
parents or guardians will impact on the placement decision. Moreover, per- 
sonnel responsible for providing special services to the student and the 
student's vocational program instructors) play a key role in assisting both 
student and parents in deciding upon the best training option for the stu- 
dent. Therefore, in selecting the most appropriate vocational education 
program for the student, it is important that all parties collectively par- 
ticipate in the placement decision-making process. 

Purpose 

The purpc. e of this module is to provide you and the CBVA team mem- 
bers with a set of assessment strategies and guidelines for determining the 
most viable vocational program placement for a student with a handicap. 
The information provided will be especially useful to personnel who are 
responsible for helping individual students and their parents in selecting 
one specific vocational education program among the many options avail- 
able in the school district 

Assessment for Placement Strategies 

The activities that occur during the placement process should provide 
the student, his/her parents or guardians and school personnel with 
answers to twc important questions (Cobb & Larkin, 1985; Cobb, 1985; 
Cobb & Albright, 1986): 

1. is vocational education the best or most appropriate 
curriculum for the student? Or, are the college 
preparatory or general track curricula more ap- 
propriate? 

2. Which vocational education program is most ap- 
propriate for the student? 
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The first question, the appropriateness of vocational education as a cur- 
ricular option, is more general and must be addressed before committing 
time and resources to the second and more specific question about selection 
of a particular vocational program direction. 

The discussion that follows is organized in terms of the assessment 
process needed to address each of the above key questions. Particular at- 
tention is given to when and where these assessments need to occur and by 
which personnel. Suggestions for conducting these assessments are also of- 
fered. 

Vocational Education as a Curricular Option 

The activities of communicating vocational education options to stu- 
dents with handicaps and their parents (Module 1, Part II) represent a 
starting point for determining if the vocational education curriculum is an 
appropriate route for the student. If, indeed, the student and parents are 
interested in the vocational education curriculum, then the vocational ex- 
ploration process continues. This process, which should begin a year prior 
to the student's eligibility for enrollment in vocational education, will re- 
quire coordinated efforts among the student^ IEP manager, vocational 
guidance counselor, and vocational special needs resource personnel. In 
order to structure this coordinated process, the following procedures 1 are 
recommended: 

1. Establish a Vocational Placement Assessment 
Team 

As suggested earlier, the members of this team should 
be comprised of at least three personnel: 

A. The individual responsible for managing the 
student's IEP; 

B. A vocational guidance counselor; and 

C. An individual who provides special needs services 
to students with handicaps in vocational t?u jcation 
programs. 

While the primary purpose of the team is to organize an 
exploratory program for the student, the team will also 
focus its attention on collecting the assessment informa- 



Adapted from Minnesota Vocational Assessment Guidebook Team (1986, 
December). Vocational assessment: A guide for improving vocational 
programming for special needsyouth. Bemidji, MN: Bemidji State Univer- 
sity, Center for Vocational Education. 
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ment. Hence, the team approach is recommended as a 
means for sharing these assessment for placement 
responsibilities. 

2. Analyze Student's Cumulative Record 

Before the first team meeting, the members should be- 
come familiar with the student's prior educational per- 
formance. One method of gaining familiarity is to 
review the student's cumulative record* Several tips for 
conducting a student records review are provided in 
Table 1. In addition, a sample form for summarizing key 
information found in a records review appears in- 
Example Document #1. 

For team members who, heretofore, have not done a 
review of records of a student with a handicap, a check 
of applicable school district policies and procedures is 
advised. This check will provide guidance on personnel 
clearance procedures for obtaining access to student 
records. 

3. Interview Student and Parents 

A designated member of the assessment team should 
schedule an intake interview with the student and her 
or his parents. The purpose of the session is to deter- 
mine the preferences of student and parents for a voca- 
tional education curriculum. If both parties are clear in 
their preference for vocational education, then discus- 
sion of specific programs of interest should follow. 

In order to provide accurate and up-to-date information 
to both student and parents, the assessment team mem- 
ber in charge of this interview will need to be very 
knowledgeable about the range of vocational education 
programs offered in the district. Sharing current infor- 
mation with parents and student about such matters as 
the entry-level requirements of individual programs, 
the expectations of the instructors), the support ser- 
vices available and the future employment prospects for 
graduates of these programs, should result in a mutual 
understanding of program options and potential direc- 
tions. 

Student interest information is often collected prior to 
the interview and used as an organizer for exploring 
particular interest areas during the interview. The in- 
take interview form shown in Example Document #2 
could be adapted for this purpose. 
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The person in charge of an interview must attend to a 
variety of verbal and non- verbal actions during the in- 
terviewing process. The checklist provided in Example 
Document #3 should be helpful to the interviewer in 
planning for an interview and in reviewing the results 
of the interview. 

4. Organize Exploratory Assessment Program 

At its first meeting, the assessment team will focus on 
three items; 

A. Review of the student's previous educational per- 
formance and his/her knowledge and experience in 
vocational settings; 

B . Discussion of student and parents' interest in voca- 
tional education; and 

C. Development of a plan for assessing student inter- 
ests in vocational education. 

The plan should outline the types of vocational ex- 
ploratory activities scheduled for the student, the as- 
sessment questions and data that need to be collected, 
the personnel responsible for implementing the plan, 
and the timelines for completion of the plan. 

A sample format for use in planning the student's ex- 
ploratory assessment program is shown in Example 
Document #4 . Additional information about types of 
student exploratory activities and the assessment data 
to be collected is provided in the discussion of the next 
procedure. 

While, at its first meeting, the assessment team should 
develop a rough sketch of the student's program plan, 
the final document should be prepared in concert with 
the student and her/his parents. Therefore, a member 
or members of the assessment team should communi- 
cate with the student and parents to: (a) further explore 
their interests in vocational education; and(b) complete 
the program plan. This activity should take place short- 
ly after the initial assessment team meeting. 

5. Implement an Exploratory Assessment Program 
Student Exploration Acti vities 

The type of exploratory assessment program developed 
for the student will depend on such factors as the 
student's knowledge about vocational education 
programs and the level of student and parent commit- 

l?« 



Purpose 



Table 1: Conducting a R 

To obtain a basic understanding of the student's prior 
academic performance, medical history and related 
vocational experience. 



Information While the amount and types of information contained 
Available in a student's file will vary among school districts. 

Salvia and Ysseldyke (1 988) describe throe categories 
of information typically found in student records: 
Category A: Which includes basic identifying data the 
school needs to operate the educational 
program, such as student and 
parents/guardians' names and addresses, 
student's age, grades completed, atten- 
dance, and achievement evaluations. 
Category B: Test results and other verified infor- 
mation useful to the student's educa- 
tional program are classified in this 
category. Items such as the following 
are within Category B: 

• Hearing and vision screening reports; 

• Medical history; 

• Results from academic, aptitude and 
interest tests; 

•Systematic observations and coun- 
selor ratings; 

Category C: Essentially represents a collection of 
unevaluated but potentially useful in- 
formation. For example, a handwrit- 
ten teacher memo, which summarizes 
a discussion with the student about 
her/his part-time work experience 
could be considered Category C infor- 
mation. 

Cautions The following cautions in reviewing records are of- 
fered (Goh & Sitlington, 1985): 

• Information contained in the records describes the 
past. The student may have changed considerably 
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of Student Records 



since negative comments were written earlier. 
Problems, academic or behavioral, may* have been 
overcome. Be cautious about first impressions 
formed on the basis of dated information. 

• View IQ results with caution, Group IQ tests may 
not be reliable for individuals. The test may have 
been biased, given in an inappropriate mode (lack 
of reading ability does not in itself indicate low in- 
telligence), or given on a day when the learner felt 
threatened, uninterested, or just ill. 

• Check standardized achievement and aptitude 
tests with equal caution. The cautions mentioned 
for IQ tests apply here as well. In addition, stand- 
ardized test results are usually too general tc aid in 
pinpointing specific strengths and weaknesses. 

• Look for any apparent trends in grades or achieve- 
ment. Changes in schools, family or health 
probl ms, and frequent absences should be com- 
pared to grade patterns for Hues to explain 
academic or benavioral problems (p. 12). 

Key Review In reviewing the student's record for vocationally 
Questions relevant information, the following questions should 
prove helpful (Albright, 1978): 

A. What specific abilities, interests and strengths of 
the student are noted? 

B. What appears to be the most significant problem 
likely to affect the student's performance in a 
vocational setting? 

C. What has been done in the past to correct these 
problems? How effective were these interven- 
tions? 

D. What types of vocationally related experiences 
are noted in the record (e.g. , participation in in- 
dustrial arts, school and/or community-based 
career exploratory experiences)? 
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ment to a specific career path. If, for example, a student 
and his/her parents are unclear as to a career direction 
and have limited knowledge of vocational education op- 
tions, then an extensive exploratory program is likely 
to be needed. Such a program may begin with a more 
general orientation (e.g., tour of vocational programs 
and facilities, discussions with vocational program in- 
structors and students) and eventually move into a 
more specific focus (e.g., tryouts in three or four 
programs in two occupational areas, shadowing 
cooperative work experience students). However, for the 
student with a clear career direction and some under- 
standing of the options available through vocational 
education, her/his exploratory program may be very 
specific at the outset (e.g., tryouts in two programs in 
one occupational area). 

Team Asses sment Needs 

Since a basic assessment consideration in an explora- 
tion program is that the student receives direct ex- 
perience in a vocational education program or 
programs, one often used method is the vocational 
tryout (Hartley & Lehman, 1988). This method has the 
student performing a skill or a set of skills in the voca- 
tional education setting. By systematically observing 
the student's performance, important assessment infor- 
mation about his/her level of work skills and habits can 
be directly obtained. In a recent publication, Hartley & 
Lehman (1988) outlined a series of steps to follow in 
designing and using vocational tryouts. These steps are 
shown in Table 2. For an illustration o c a completed 
vocational tryout assessment procedure, see the 
agriculture education performance sample displayed in 
Example Documen t #5. 

Which Vocational Program? 

The student exploration and team assessment activities conducted as 
part of the exploratory assessment program will produce the information 
necessary to determine student readiness for entry into vocational educa- 
tion. At this point, a decision about student entry into a specific vocation- 
al program should now be possible. When narrowing in on a particular 
program option, the assessment team should address thj following 
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Table 2: Organizing a Vocational Program Tryout 



i 



.Purpose 

Assessment 
Personnel 

Steps 



2. 



To assess student performance on tasks related directly to the 
requirements of a vocational education program. 

Member(s) of exploratory assessment team (e.g., special needs 
teacher and/or vocational instructor). 

1. Ask the vocational instructor what skills are required for 
students entering the program.- 

Ask the instructor to choose one joh/skill taught in the vocation- 
al program that is representative of the area. Skill must be 
hands on, realistic and related directly to the curriculum. Tryout 
should have a designated completion time (2-3 hours). 

List entry level skills/competencies that the student will 
learn during the tryout.If the vocational program area re- 
quires academic and job related skills include an assessment 
of both in the tryout. 

Brainstorm with vocational instructor some activities which 
would demonstrate the student's mastery of some of the 
above skills.List the following assessment information: (a) 
setting; (b) materials/ equipment needed; (c) tasks to be per- 
formed; and (d) evaluation standards. 

Write an assessment plan which includes the following sec- 
tions: (a) title; (b) materials/equipment needed; (c) materials/ 
equipment identification sheet; (d) definitions of vocational 
vocabulary students should know in order to complete the 
assessment; (e) directions for performing the tryout (task 
analysis of the required steps); and (0 measures for evaluate 
ing student performance. 

Additional considerations: 

• Cover safety aspects of vocational area; 

• Introduce student to instructor; 

• Give student a tour of facility; explain rules; 

• Supervise activities closely; 

• Make sure assessment is hands-on; 

• Observe student's tolerance, frustration, fatigue and com- 
fort levels. Make adjustments accordingly; 

• Observer(s) must monitor student progress and give im- 
mediate feedback to the student; and 

• Obtain student feedback on experience gained through the 
tryout (e.g. interest level, skills learned). 
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manual (pp.20-23). Greeley, CO: University of Northern Colorado, Vocational 
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program-specific questions (Cobb & Larkin, 1985; Erekson, 1981; Cobb & 
Albright, 1986): 

1. What types of support services are available to assist 
the student in the vocational program? 

2. What is the teaching style of the vocational instruc- 
tors)? Cobb and Larkin (1985) suggest examining 
whether the instructor uses such teaching strategies as 
peer teaching and peer tutoring, multiple media in lec- 
ture, and logical, well- sequenced presentations. 

3. Is the program of instruction offered in one location 
(e.g., in-school lab) or in multiple locations (e.g., lab and 
community worksite)? What access problems may arise 
for a student with a physical handicap? Or, for the stu- 
dent in need of a structured instructional environment, 
what accommodations will need to be made in a program 
that includes a loosely structured arrangement at a 
community worksite? 

4. Does the vocational program include a cooperative work 
experience in the community and/or an entrepreneurial 
experience in school (e.g., working at a school store as 
part of a distributive education program)? The oppor- 
tunity to use vocational skills taught in simulated set- 
tings in actual work settings is a desirable program 
feature. 

The Concluding Step 

The final step in the placement decision-making process is for the as- 
sessment team to share its information with the parents and student. The 
outcome of tins session should be a recommendation for enrollment in a 
specific vocational program. Moreover, the assessment information col- 
lected during this program placement phase will be helpful in the next 
phase of programming for the student; which involves the development of 
a special services plan for the student in a particular vocational education 
setting {Module 4). 
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Summary Report of Student's Cumulative Record 

Student Reviewer 

Date of Review 

I. Category A; Basic Identifying Information 
Student Address: 

Parent Names and Address: 

Educational Progress Data: 

II. Category B: Test Results 
Medical Data: 

Results: 

Impressions: 

Achievemen t/Aptitude/In teres t Test Data: 

Results 

Impressions: 

III. Category C: Other Information 
Pertinent Observations: 

IV. Overall Comments: 
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Initial Intake/Interview Form 1 

Date: 

Name: . 



Address: 



City: State: Zip: Phone: 

Parent or Guardian (if minor): 

Address (if different): ' 



City: State: Zip: Phone: 

School: 



Date of Birth: 



Ethnic Background: 
Family Status: 



Explain any of the following with which you have difficulty: 
Vision: 



Hearing:. 
Lifting: _ 



Sitting or Standing Tolerance: , 
Medications: 



Physical Limitations: 
Stress Tolerance: 



Learning Problems:. 



^rom Lehmann, J. (1985, July), Vocational assessment handbook. Greeley, CO: University of Nor- 
thern Colorado, Vocational Teacher Education Department 
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Vocational Interest (Goals): 
Career or Job In Which You Are Interested 

Work and Life Experiences: 

Current Job: 

Full Time: P art Time: 

Current Duties: 



Past Job Experiences: 



Volunteer Experiences: 



Other Work-Related Experiences: 



Likes and Dislikes in Work Experience: 



Special Problems in Employability: 



Special Skills and Talents: 
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Educational Background 

Highest Level Completed: 

8th Grade 

High School 

GED 

College— Number of Years: 1 2 3 4 

Name of Schools Attended: (Last one attended first): 



Special Training: 



Problems or Difficulties Encountered in Education: 



Favorite Subjects: 



Problems Anticipated in Planning Future Education: 



Time Available to Train in Future: 

How Long: 

Resources (Funds) Available: 



Do you have transportation? 

Method: 

Sources of Financial Support: 
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Interviewer Checklist 1 

During the interview I performed the following behaviors: 
Yes No N/A 

1. Put aside what I am doing and give the person my 

total attention. 

2. Establish a proper distance between me and the 

interviewee. 

3. Face the person squarely. 

_ 4. Lean or rotate my body toward the person. 

5. Develop and maintain eye contact. 

6. Assume and maintain an "open" posture. Do not 

cross arms and legs. 

7. Maintain a relaxed position. 

8. Refrain from distracting movements (fidgeting, 

etc.) 

9. Observe the person's posturing, 

10. Observe the person's personal appearance. 

11. Observe and determine the person's level of ac- 
tivity. 

12. Observe the person's facial expressions. 

13. Look for and determine the person's feelings from 

the person's non-verbal communication. 

14. Look for patterns in the person's non-verbal com- 
munication. 

15. Develop hypotheses based on the person's non- 
verbal cues. 

16. Resist internal di factions. Focus my total atten- 
tion on the person and not think about extraneous 
things. 

17. Initially suspend judgement. I show none of the 

evidence of judgmental thinking such as inter- 
rupting the person, obvious affective reactions, 
etc. 



From Farley, R.C. & Rubin, S.E. (1980). Systematic interviewing skills: Participant's workbook Fayetteville, 
AR: University of Arkansas, Arkansas Rehabilitation Research and Training Center. 
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Interviewer Checklist (continued) 

Yes No N/A 

18. Have goals for listening, I know exactly what I am 

listening for and listen systematically, 

_____ 19, Listen to the actual words of the person that 

describe the situation he/she is talking about I 
hear exactly what the person says he/she sees, 
hears, senses, etc. 

20, Listen for a description of what the person says to 

self nbout the situation being discussed. 

21. Listen for and determine the person's feelings 

from his/her verbal expressions. 

22, Listen for the person's description for his/her be- 
havioral actions in the situation. 

23, Listen for and focus on the person's mqjor area 

concern. 

24. Listen for common themes in the person's verbal 

expressions. 

25. Listen to person's tone of voice; volume, intensity, 

rapidity, etc. 

_____ 26. Am aware of any inconsistencies in the person' 

communication especially between verbal and 
non-verbal communication. 

27. Communicate my attentiveness by varied facial 

expressions, smiling, head nods, hand gestures, 
etc. 

28, Wait before responding verbally. 

29. Systematically organize, process, and label what 

the person has seen and heard in a facilitative 
manner. 
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Vocational Exploratory Assessment Plan 



Student Name 



Date 



Team Members 



(IEP Manager) 



(Guidance Counselor) 



(Vocational Special Needs Teacher) 

I. Student and Parent 
Interest Data: 



IL Vocational Exploration 
Assessment Activities: 



IIL Recommended Vocational 
Program Placement: 



Activity 



Assessment 
Date 



Strategy 



Personnel 
Responsible 



Example Document #5 
An Agriculture Education 
Performance Sample 1 
(Horticulture) 



By 

Linda Falk 
Anita Knight 
Jack McStott 
Herbert Shahan 
Clover Park School District 



From Gill, D. & Associates. (1986, November). The vocational education assess- 
ment plan, pp. 87-94. Tacoma, WA; Educational Service District #121, 
Pierce County Vocational/ Special Education Cooperative. 
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Instructions for Administering 
the Nursery /La ndscape Performanc e Sample 

The performance sample should be administered by the greenhouse in- 
structor or someone trained and/or experienced in the greenhouse area* For 
increasing the reliability of observations, it is recommended that more than 
one person observe and record student performance, 

Materials Needed: 

Various Pots - 2", 4", 6" Peat Moss - 1 quart 
Geranium Plants Rooting Hornk ^e Powder 

Sand - 1 quart Instruction Sheet 

Equipment and Tools Needed: 

Hand Trowel Sharp Knife 
Bucket - 1 quart Clear Plastic Bag - Twist Tie 
Measuring Cup - 1 quart Scissors 
Ruler Protractor (flat plastic) 

Shallow Dish Thermometer 

Items Needed by Individual Administering Performance Test 

Evaluation Form Instruction Sheet 

Timing Device Pencil or Pen 

Before starting the student on the performance sample, the ad- 
ministrator should review with the student(s) the material and equipment 
needed. Also, the administrator should inform the student that she or he 
may ask questions at any point during this exercise. If a student has dif- 
ficulty reading the instruction sheet, the individual administering the test 
should explain the instructions in detail. Likewise, if a student cannot read 
the drawings, an actual sample should be furnished. The student is now 
ready to begin the performance test. 

The administrator is to observe the student during the performance, 
while recording on the checklist provided. In addition to the items noted on 
the checklist, the individual administering the test may find it helpful to 
record the type and frequency of questions asked by the individual(s) taking 
the test. This could indicate some area of difficulty in the test conditions. 
After the completion of the performance test, the student's finished product 
should be checked for accuracy by the individual administering the test. 
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Instructions for Student Completing 
Nursery/Landscape Performance Sample 

Rationale 

As a student planning to enroll in the Nursery/ Landscape program, you 
are to read the following directions and complete each step as directed. This 
test will give you an opportunity to demonstrate the skills you already pos- 
sess, as well as identify the areas you may need help in while enrolled in 
Nursery/Landscape. The skills used in completing this performance test re- 
late to a great majority of the tasks you will be involved in while complet- 
ing your Nursery/Landscape program. 

This test is being given to you to help plan your program to be of max- 
imum benefit to you. It is not used for screening students from the program! 

Materials Needed 

Various pots - 2", 4", 6" Peat Moss - 1 quart 
Geranium plants Rooting Hormone Powder 

Sand - 1 quart Instruction Sheet 

Equipment and Tools Needed 

Hand Trowel Sharp Knife 
Bucket - 1 quart Plastic Bag - Twist Tie 
Measuring Cup - 1 quart Scissors 
Ruler Protractor (flat plastic) 

Shallow Dish Thermometer 

Objectives of this activity 

Upon completion of this activity, you will have demonstrated the fol- 
lowing skills: 

• Ability to follow directions 

• Thermometer reading 

• Measurement skills: linear, dry and liquid 

• Identification of basic plant parts 

• Recognition of angles 

• Sequencing skills 

14 fi 
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Steps: 

First, check to see that the greenhouse temperature is set at 72 degrees. 

Second, check to see that all needed materials are available. 

Third, read each step carefully before you bogin the performance 
sample. 

Procedure Sheet 

Step 1. Review the complete procedure sheet. 

Step 2. Into a small plastic bucket, measure 1 pint of peat moss and 

1 pint of sand. Stir and mix completely with a small hand 
trowel. 

Step 3. Select a 4 inch pot. 

Step 4. Fill the pot with plant mixture and firm the plant mixture 
down so that the surface is 1/2 inch below the top edge of the 
pot. 

Step 5. Study the plant diagram carefully. 

Step 6. Cut an 8 inch long piece of stem from the top of the geranium 
plant. 

Step 7. Place the cut stem down on the table. 

Step 8. Place the protractor along the side of the stem. Cut a 4 inch 
piece from the top of the stem at a 30 degree angle (with a 
sharp knife), making the cut just below a leaf. 

Step 9. Remove all the leaves on the lower half of the cutting. 

Step 10. Dip the cut end of the stem into the rooting hormone pow- 
der. 

Step 11. Lightly tap off any excess powder. 

Step 12. In the center of the pot, insert the cutting to about 172 its 
length in the plant mixture. 

Step 13. Pour 1/3 cup of water into the pot and allow it to drain. 

Step 14. Place the pot in a plastic bag (see diagram), and tie the bag 
tightly at the top. 

Step 15. Clean up your work area. 
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PLANT DIAGRAM 
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Student Performance Evaluation Form 

Student's Name Date 

Observer(s) 

Time StartedL^^ Time Completed 



Test Performance 



Task Sequence 


Accurate 


Independent 


With Assistance 


1. 


Read thermometer 








2. 


Measure 1 pint of sand 
and 1 pint of peat moss 








3. 


Mix plant mixture 
completely 








4. 


Select 4" pot 








5. 


Fill pot, firm mixture to 
within 1/2" of top 








6. 


Measure and cut 8" stem 








7. 


Measure and cut 8" stem 








8. 


Remove all leaves on 
lower half of stem 








9. 


Apply rooting hormone 








10. 


Place cutting in pot 








11. 


Measure 1/3 cup of water 








12. 
13. 


Wrap plant in plastic bag 
Clean work area 
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Module 4 
Planning the Student's 
Vocational Education Program 



GOAL: 



COMPETENCIES: 



Upon completion of this module, you will gain 
an understanding of the knowledge and skills 
necessary to assess the vocational curriculum 
anaVor the instructional environment such that 
interventions can be planned and incorporated 
into vocational IEP development. 

In order to plan a student's vocational educa- 
tion program, you should demonstrate: 

1 . Understanding of the distinction between 
interpreting assessment data for program 
placement purposes, and interpreting 
data for program planning purposes; 

2. Understanding of eligibility determina- 
tion for supportive services in regular 
vocational education programs; 

3. Knowledge of the types of assessment 
questions/issues that are resolved through 
program planning assessment processes; 

4. Skill in conducting content-specific sur- 
veys; 

5. Understanding of how to generate class- 
room-specific norms, and their utility in 
eligibility determination and program 
planning; and 

6. Skill in completing a vocational program 
inventory, and using it to improve instruc- 
tional remediation. 
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Introduction 

In Module 3, you learned some techniques to assist in helping a stu- 
dent, his/her parents, and school professionals in deciding whether or not 
vocational education was the proper curriculum choice, and, if so, which 
particular vocational program area. These activities would occur prior to 
the time when the student would actually start vocational classes. 

Module 4 builds on the assessments conducted in Module 3, but differs 
in two fundamental ways: 

1- Timi ng. Assessment activities, including analysis and 
interpretation of prior assessment data and use of new 
forms of assessment instrumentation, are conducted 
after the student starts her/his vocational class. 

2. Purpose . Instead of determining vocational program 
area, data analysis and interpretation are used to deter- 
mine eligibility for supportive services for the student, 
and the nature and intensity of those services. 

Figure 1 below graphically clarifies these two distinctions. As can be 
seen, assessment for program planning differs first in its timing. Students 
will most likely have been assessed and placed in vocational programs in 
the Spring semester prior to their enrollment in the vocational program. 
At that time, broad annual goal statements may have been developed for 
the IEP which begin to focus on the areas of remediation which will be re- 
quired and the types of support services that the student will need during 
the program (Albright, 1978). Assessment for program planning, the focus 
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of this module, occurs during the first three to six weeks of the student's 
enrollment in his/her vocational program (typically at the start of the Fall 
semester). At this time, it will be determined if the goal areas identified in 
the Spring are indeed critical domains, given the environmental and cur- 
ricular contingencies of this particular vocational program. 

This review of the broad annual goals determined in the Spring 
semester suggests the second critical difference between the two sets of as- 
sessment activities: purpose. Federal vocational education law states 
that a student who is on an IEP is not eligible to receive special services in 
vocational education unless his/her handicapping condition interferes with 
successful performance in his/her specific vocational education program. 
Thus, eligibility for special education services is a necessary but not suffi- 
cient criterion to determine eligibility for specialized services in vocation- 
al education. A student must not only bp identified as handicapped (using 
special education eligibility criteria), but that handicap must also interfere 
with that student's success in vocational education. For example, a student 
labeled behavior disordered who was "turned off' by academic course work 
might function perfectly well in a vocational education program. That stu- 
dent would not be eligible for special services (nor would he/she need them) 
in vocational education even though she/he was considered handicapped in 
academic coursework. Similarly, a student who experienced reading com- 
prehension problems in English classes in the 9th and 10th grades might 
not experience the same level of difficulties in his/her vocational program 
during the 11th grade. This might be the case because the vocational 
teacher placed less emphasis on reading comprehension on her course, or 
because the nature of the reading material increased the motivation of the 
student to make an extra effort to absorb it. The important point here is 
that a student's skill levels and the need for additional support are, at least 
in part, determined by the curricular context in which he or she is asked to 
demonstrate them. Thus, the determination of the need for specific support 
services, and the nature and intensity of these services, needs to occur after 
placements have been determined and the student has had a chance to in- 
teract with the curriculum. 

Timing and purpose aside, a final distinction between assessment for 
placement (Module 3) and assessment for program planning (Module 4), 
relates to the level of specificity of what is being assessed and how the data 
are interpreted. Ir jessing students for placement, support personnel are 
interested primarily in assessing the student for his/her interests and 
abilities relative to the various vocational program options. With assess- 
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ment for program planning, however, it is equally important to assess the 
curriculum, the style of instructional delivery, the laboratory equipment 
and facilities, and other domains, to look for areas where student perfor- 
mance in the instructional environment suggests a need for some form of 
supportive assistance to assure a successful experience. To be sure, these 
kinds of domains may have been considered during the placement process, 
but not at the level of detail that must occur in order to efficiently target 
supportive assistance. Thus, assessment for program planning involves a 
careful examination of student performance while he or she is interacting 
with a particular vocational curriculum. Placement assessment data, such 
as information from prior testing, interview data, and observations on per- 
formance samples, may be used for this purpose. However , it is likely that 
additional types of data will be required in order to bring the focus of the 
assessment from the broad, annual goal orientation to a more fine-grained 
instructional objective orientation. 

Purpose 

The purpose of this module is to provide you with a series of strategies 
that can be used to assess student/curriculum interaction for the purposes 
of determining eligibility for supportive services, and for planning the na- 
ture and intensity of a student's support needs, The final outcome of this 
assessment will be to generate information to assist special support per- 
sonnel in listing the student's short-term instructional objectives for the 
vocational component of his/her IEP, along with reasonable criteria along 
which progress towards these objectives can be monitored. 

Assessment for Program Planning Strategies 

The activities that occur during the assessment for program planning 
process should provide the student, his/her parents or guardians, and 
school personnel with answers to the following questions (Cobb & Larkin, 
1985; Cobb, 1985; Cobb & Albright, 1986): 

1. Is this particular student eligible to receive support ser- 
vices in his/her particular vocational education 
program? 

2. In what particular areas of instructional remediation 
should support personnel focus to ensure success for a 
student in his/her vocational program? 
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3. What is the natuis and intensity of support services 
necessary to deliver this instructional remediation? 

4. What criteria should be applied to a student's perfor- 
mance which reasonably reflects the success norms of 
the vocational program? 

To adequately address these questions, vocational support personnel 
will need to conduct two types of assessments: 

1. An assessment of the student's performance relative to 
the requirements of the vocational program; and 

2. An assessment of the environmental conditions sur- 
rounding a student's performance in a vocational 
program. 

This module is organized around a discussion of these two require- 
ments, and two types of assessment processes which can accommodate 
them: 

1. Content-specific survey; and 

2. Vocational program inventory. 

Each of these two types of assessments will yield information to address 
all four of the assessment questions delineated above. 

Content-Specific Survey 

Once a student with a handicap has been placed in a vocational educa- 
tion program, has had broad annual goal areas defined for him/her through 
that placement process, and has started the actual coursework, an assess- 
ment process must occur which determines eligibility for services, and 
which targets areas for specific remediation. It is not difficult to see that a 
curriculum-based approach to this process is of utmost importance, since 
remediation can have relevance only if directly related to the vocational 
curriculum. It should be equally evident that no single instrument will be 
adequate to use in this assessment process, since students will need to be 
assessed across a wide number of domains in both academic and perfor- 
mance-related areas. What is needed, then, is an assessment process that 
will guide you in selecting/adapting commercial tests, or developing your 
own tests to measure student performance relative to vocational program 
expectations. 

To assist you, a six-step process has been outlined below, which has 
been adapted from Zigmond, Vallecorsa, and Silverman (1983) and which 
will fulfill these functions. This process is a simple, yet powerful procedure 

15a 
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designed to produce accurate, curriculum-lased measurement of student 
skills, and, over time, a set of program-specific criteria (norme ) that stu- 
dents should be expected to achieve. 

Step L Decide What to Assess. 

You must first decide what curricular domain(s) need to be assessed. In 
many cases, basic academic skills, such as math and reading, will need to 
be surveyed, as well as such generic areas as note-taking, test-taking, skim- 
ming and scanning, textbook usage, and skills in understanding 
schematics. In addition, general laboratory safety, performance in gross 
and fine motor operations, social behavior, and knowledge of specific entry- 
level competencies unique to the curriculum may have to be assessed. What 
is critical here is that you as a vocational resource person or a laboratory 
teacher have an accurate understanding of the variety of domains neces- 
sary for assessing in each of the vocational program areas. While the poten- 
tial list of domains is extensive, research by Greenan (1986) indicates that 
four areas (math, communications, interpersonal relations, and reasoning 
skills) comprise the major areas that will need to be assessed. 

Step 2: Select or Develop the Survey Instrument 

You will now have to either select a commercially available test, or 
develop your own instrument to assess the particular area of interest. There 
are advantages and disadvantages to either selecting or developing a sur- 
vey instrument. If commercially available instruments are selected, there 
appear to be several advantages: 

1. Instrumentation will typically have a validity and 
reliability base; and 

2. Formality of standardized test increases face validity of 
instrument with students and teachers, and students 
and teachers may take the administration and inter- 
pretation of results more seriously. 

Disadvantages of using standardized instrumentation include: 

1. They are usually more expensive to buy, and often are 
costly to score and interpret; 

2. Frequently, items on the test will not represent cur- 
riculum content, while others will, causing students to 
take time to complete some test items which are not 
relevant; 

ERJC 
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3. Standardized instruments can be intimidating for stu- 
dents to complete; and 

4. Often, test administration guidelines are inflexible and 
work against students performing at maximum ability. 

The other option is for vocational resource teachers to develop their own 
informal survey. Teachers can choose parts of standardized instruments, 
create their own instruments, or some combination of both. Although these 
informal surveys can take some time to develop, and lack formal validity 
and reliability information, teacher-made informal surveys have some dis- 
tinct advantages: 

1. Their content can be tailored directly to vocational 
program content and requirements; hence, item content 
is efficient and exact; 

2. The process of developing the instrument can help 
resource personnel learn more about vocational 
program content; and 

3. The administration, scoring, and interpretation of the 
instrument can be tailored to the unique needs of each 
student completing the survey. 

When selecting a particular commercially-available test for adapting to 
your own vocational programs, be sure that it does indeed measure the 
exact skills important for success in your vocational education programs. 
One often-used way of doing this is to verify the importance of the test con- 
tent with vocational instructors in your district. To illustrate these points, 
consider the generalizable skills and associated tests developed by Greenan 
(1986). Since the various generalizable skills (115 in total) were validated 
by vocational educators in many secondary schools, one can safely assume 
that these skills are representative of most vocational programs. While 
generally true, you will need to know which, among the many generalizable 
skills, are especially important to vocational personnel in your district. By 
obtaining this information, you can focus your testing on only those skills 
of primary importance in your district. Such activity will streamline your 
testing procedure and produce information valued by district vocational 
personnel. 
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Regardless of whether you select or develop your survey, several impor- 
tant questions will need addressing in order to make the use of this survey 
most effective: 

1. Which specific skills are important for each vocational 
class? Even if a vocational resource teacher develops 
his/her own mathematics survey for the vocational cen- 
ter, for example, some requirements of the survey will 
be more germane for some classes than for others. Thus, 
the resource teacher will have to review the math sur- 
vey with each teacher and determine which sub- 
domains of mathematics are important for his/her class. 

2. At what skill level must students be able to perform in 
order to successfully function in the class? For example, 
should students be able to change fractions to decimals 
with 60%, 75%, 90%, or 100% accuracy to be able to learn 
course content successfully? By ascertaining the skill 
level necessary in each program, the vocational resource 
teacher can begin to develop classroom-level norms 
which can be extremely valuable in determining if a stu- 
dent needs remedial help, and at what level remedia- 
tion needs to occur in terms of establishing realistic 
criteria for success in areas of remediation. 

3. At what cognitive level must students be able to perform 
in order to function successfully in class? For example, 
a student may be able to divide 3/4 by 1/2 correctly at a 
computational level, but when asked to solve problems 
using the same skill (i.e., "halve a recipe" in a Foods Ser- 
vice Class), she may not understand how to apply these 
computational skills. Vocational resource teachers 
must establish the cognitive level necessary, and sur- 
vey students at that level to accurately assess areas for 
remediation. 

By responding to these three issues, you are able, first, to establish a 
setof norms, or minimum competency levels, around which eligibility deter- 
mination for support services can occur. For example, experience may in- 
dicate that students in Architectural Drafting who score below 75% on 
converting fractions are likely to be in serious difficulty in the remainder 
of the course. This could be the cut-off for eligibility determination in this 
vocational program. However, in some other course, the emphasis on frac- 
tional conversion might be less important, and hence this subtest would 
not be relevant at all in eligibility determination. 
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Step 3: Get Ready to Test 

Often, the most important variable in ensuring that a student will per- 
form at his/her best on a content-specific survey is to make sure that the 
test environment is best suited for the student to demonstrate his/her skills. 
Considerations here include independent, one-on-one, or group administra- 
tion; special equipment needed to complete the survey; knowledge by the 
student concerning how to complete the survey; and a quiet, comfortable 
surrounding in the test environment. As Zigmondet al. (1983) stated: 

"Being prepared, thinking ahead, and testing in an organized, 
well-managed environment does much to put the student at ease 
and elicit his/her best performance." (p. 38) 

Step 4: Administer the Survey 

Vocational resource teachers must recognize at the outset of this sur- 
vey process that it is not necessarily the score that is important, but the 
student's performance that counts. Thus, teachers should encourage stu- 
dents to cross-out unwanted responses, rather than erase them, so that 
error patterns can be examined at a later date. Similarly, students taking 
tests orally should be tape-recorded in order to have a permanent record of 
how a student arrived at a correct or incorrect response. Finally, toachers 
should be prepared to alter test directions, add test-taking incentives, or 
do whatever else they have to do to get the maximum performance from a 
student. 

In many cases, noting the student's interaction with the laboratory 
equipment, as well as performance with the test, will assist in interpreta- 
tion of results and the most effective targeting of supportive service s. It is 
also extremely important to consider what occurred to the student just prior 
to the test-taking process, and the events anticipated to occur immediate- 
ly after testing. Often, knowledge of these earlier and subsequent events 
will assist in later interpretation of patterns of errors. 

Step 5: Analyze Findings and Summarize Outcomes 

It is impossible to overstate the importance of this step in the survey 
process. Again, it is not a score that is of importance in this step, rather an 
analysis of the types of errors that have been made. It should be remem- 
bered that this is a relatively broad survey of a particular domain, such as 
math or vocabulary skills, and often there will be only a small sample of 
items around a particular skill. For example, in a math survey, there might 
be 5-6 items each in the skill areas of whole number manipulation, frac- 
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tional transformation, measurement, percentages, geometric skills, and 
metrics. In this step of the survey process, vocational resource teachers will 
be examining correct and incorrect responses, and will be determining at 
what cognitive level (computational versus problem-solving), or at what 
skill level (mastery, partially learned, or skill not known at all) the student 
is functioning. 

In examining incorrect responses, the vocational resource teacher must 
first determine that errors were not a result of disinterest or other vari- 
ables unrelated to the skills being tested, such as a poor night's sleep, 
anxiety over a class that followed the survey, or a fight with a friend just 
prior to the survey. If these kinds of extraneous variables appear to have 
influenced error patterns, results of the assessment should be discarded 
and arrangements made to redesign the testing environment daring a 
retest. If, however, it is assured that the assessment reflects motivated per- 
formance, then Zigmond et al. (1983) suggest two alternative hypotheses 
for student errors: 

1. Errors made because, although the skill was known, the 
test conditions prevented a demonstration of the skill; 
or 

2. Errors were made because the skill was not known. 
In the first instance, several reasons might account for errors: 

1. Skill was known but test item was too complex. 

Example: The intent of the test item was to determine if the stu- 
dent knew how to change a fraction to a decimal. The 
item was "142/275." The student might have known that 
you divide 275 into 142, but made errors in the computa- 
tional portion of the skill. Thus, the student knows the 
process of changing a fraction to a decimal, but does not 
know complex long division. 

Next Step: Develop a probe with less complex fractions and deter- 
mine precisely where computational problems with the 
long division process occur. 

2. Skill was known but response mode was inappropriate. 

Example: Survey items show a picture of a hand tool, and the stu- 
dent is asked to write the name of the tool and a use for 
it. The student might actually know the tools and a use 
for them, but is unable to write the names and u^eb. 

Next Step: Develop a probe with a different response mode, i.e., 
have the student verbally name the tool and tell a use 
for it. 
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3. Skill was known but response level was too difficult. 

Example: Survey item includes a schematic of a carburetor and 
student is asked to write the parts. The student might 
be able to associate parts of the carburetor if he/she had 
a list of parts to work from. 

Next Step; Develop a probe at a lesser response level, i.e., have the 
student match the names to the parts of the carburetor. 

Alternatively, the student might not know the skill. Two alternatives 
exist in this instance; 

1. Student required cues or prompts to complete the item. 

Example: Survey items asked the student to identify the 

important concepts in a segment from chapter 
of a textbook. Student repeatedly asked the 
teacher if different elements of the chapter seg- 
ment represented important concepts, such as 
italicized words and underlined words. 

Next Step: Develop a more detailed probe with teacher 

prompts which identifies more clearly which 
skills the student has mastered and which 
he/she has not. 

2. Student does not know skill at any level 

Next Step; Develop a probe at a lower level in the skill 

hierarchy to find the maximal level of the skill 
the student does know. 

The outcomes of this step in the assessment process are to firmly estab- 
lish the student's skill acquisition and the mastery level of that skill ac- 
quisition, and to identify what areas need remedial assistance* 

Step 6: Complete Record-Keeping Forms and Generate Teaching 
Objectives 

It is essential to maintain up-to-date records of student performance on 
the survey to document where he/she currently functions, whether or not 
the skill is known and at what level, and with suggestions as to how to begi a 
remediation. The objectives generated through the process can be used in 
the vocational component of the IEP. 

This six-step process can be extremely valuable in clearly identifying 
areas of skill deficits and reasonable success criteria for students enrolled 
in vocational education programs. Two researchers in vocational education, 
Dr. James A. Dunn of Cornell University and Dr. James P. Greenan of Pur- 
due University, have published a number of content-speHfic surveys as a 
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part of their research on basic skills testing in vocational education. Table 
1 on the following pages provides some essential information about the 
work of these two researchers, and how you can go about acquiring their 
specific tests. 

Vocational Program Inventory 

While the content-specific survey can assist in generating classroom- 
specific norms, in determining eligibility for supportive assistance, and in 
focusing on exactly what skills need remedial support, it is limited to cur 
ricular content areas such as mathematics and vocabulary. The Vocation- 
al Program Inventory shown in Example Document #1 is an informal 
instrument which can help you focus on the manner in which instruction 
is delivered, and thus suggest additional areas where remedial assistance 
may be needed. As can be seen, this Inventory is completed for each voca- 
tional program area, and keys on: 

1. The predominant manner in which information is 
provided to students; 

2. The predominant manner in which information is 
solicited from students; 

3. Other student classroom skills that are important; and 

4. Other student behavioral skills that are important. 

You would then match the most important or frequently used require- 
ments of the vocational program with the particular skills of a student who 
has been determined eligible for services in that program. The outcome of 
this matching process could be a change in the typical instructional proces- 
ses in the classroom to accommodate the student, or some instructional 
focal areas upon which you would want to concentrate in addition to one or 
more targeted academic areas. 

Summary 

This module has keyed in several processes that can help you determine 
if a student is eligible for services in a vocational program area, how to iden- 
tify the skills in need of remediation, and how to best deliver instructional 
support. There are certainly other techniques which can be used to assist in 
thess assessment activities, such as interviews with former teachers, the 
student's parents, and the student. One other particularly powerful techni- 
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Table 1: Selected Sources for Content-Specific Tests in Vocational Education 



The two sources described in this table were selected for two reasons: 1. they were empirically tested in, and tied directly to vocational education 
programs; and 2. being in the public domain, they can be purchased at a reasonable cost, 



Title 


Primary 


Description 


Availability 


Author 






1. Cornell Test of 


James A. Dunn 


lilcVvUIIlcll X cat UI Oct bit OKI lib \vj X DO/ lb 


ror runner intormation aoout tne 


Basic Skills 


Cornell Institute for 


a uia^iiuaLiL teaiing pruyraiii IUi iunipcien~ 


materials, wnie to. 




Occupational 


PV in Haqip mntViPTTiiifiPQ anH rparlincr TKo 

\j j in UUOIL IliatllCII latino cl 1 1 LI l\sClUlPIKl» XII" 






Education 


test battery consists of three parts: 


James A. Dunn 




Cornell University 


Form A-l, Arithmetic Computation; 


Education Department 




Ithaca, New York 


Form B-l, General Mathematics; and 


Roberts Hall 






Form C-l, Reading Comprehension 


Cornell University 








Ithaca, New York 14853 






Each test can be administered fully in one 


or 






class period. In addition to CTBS, the 


Council for Cooperative Research 






authors have produced two related docu- 


315 Elm wood Avenue 






ments: 


Ithaca, New York 14850 






1. Resource Guide for Teaching Basic 








Skills Through Vocational Education 








and 








2. Teacher Guide for Teaching Basic 








Skills Through Vocational Education. 
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Table 1 (Continued) 



Title 


Primary 


Description 


Availability 




Autiior 






2. Generalizable 


James P. Greeran, 


A set of 1 15 basic skills identified and 


Generalizable skills products can be pur- 


Skills in 


Vocational Education 


validated in vocational education programs 


chased from: 


Vocational 


Purdue University 


with assessment tests to measure student 




Education 


West Lafayette, 


skills in four areas: 


Curriculum Publications Clearinghouse 




Indiana 


1. Mathematics; 


Western Illinois University 






2. Communications; 


Horrabin Hall #46 






3. Interpersonal relations; and 


Macomb, IL 61455 






4, Reasoning skills. 


Phone:(309)298-1917 






A listing of these skills and an extensive ex- 








amination of the use of generalizable skills 








in vocational education appeared in the 








Fall 1986 issue of the Journal for Vocation- 








al Special Needs Education. 
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que is direct observation of student performance. While this technique could 
have been introduced here, it appeared most appropriate to give it more 
thorough attention in another module. Hence, the next module (Module 5), 
which is concerned with monitoring student performance, will focus in detail 
on the technology of observation. 
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Vocational Program Inventory 1 



Program Title 



Instructor 



Textbook Title/Author 
1 = Used Often 



INFORMATION INPUT 

(Instructional Methods) 

Information Sources 



Reading Level 

2 = Used Less Frequently 

INFORMATION OUTPUT 

(Types of Assignments) 

Test Format 



.Textbook 
.Worksheets 
.Lecture 
, Discussion 

v* material 
.Audio tape 
.Concrete experience 
Other: 



Short answer 
Essay 

Multiple choice 

True -false 

Matching 

Computation 

Word problems/math 

Other: 



Structure 



.Directed 
.Independent 
.Peer Tutor 
.1-1 adult 
.Small group 
.Large group/class 
Other: 



Assignments 

Worksheets 
Short papers 
Term papers 
Demo/lab projects 
Oral reports 
Group discussion 
Computation 
Word problems/math 
Maps, charts, graphs 
Other: 



Grading Criteria 



Extra Credit 



Adapted from Model Resource Room Project, Plymouth, MI, 1981. 
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Program Inventory (continued) 



Other Program Requirements 
(Check only high priority requirements) 



Academic skills needed 



Behavioral Skills Needed 



..Becoming interested 

.Paying attention to the spoken word 

.Paying attention to the printed word 

.Following directions 

.Keeping track of materials, assignments 

.Staying on task 

.Working in groups 

.Working independently 

.Learning by listening 

.Expressing him/herself 

.Reading textbooks 

.Reading study sheet or tests 

.Understanding what is read 

.Writing legibly 

.Expressing him/herselfin writing 
.Spelling 

.Seeing relationships 
.Understanding cause and effect; 

anticipating consequences 
.Drawing conclusions/making inferences 
.Remembering 
.Notetaking 
.Outlining 

.Independent research 
.Measuring 
Other: 



Coming to class on time 
Coming to class prepared 
Following direction s 
Staying in seat 
Staying on task 
Understanding/following 

safety rules 
Asking questions or for help 

help when needed 
Working in groups 
Working independently 
Other: 



Other Prerequisites. 



Attendance Policy 



Homework Policy 



Other Program Rules 



Make-up Policy. 



1.7 i 



Assessment of 
Students with 
Handicaps in 
Vocational Education: 
A Curriculum-Based 
Approach 



Understanding Curriculum- 
Based Vocational 
Assessment: Purposes 
and Characteristics 



Establishing a Curriculum- 
Based Vocational 
Assessment Process 




2 



Placing Students in 
Vocational Education 
Programs 




Planning a Student's 
Vocational Education 
Program 



3 



4 



MM- 



Planning Transitional 
Services 



Evaluating the Curriculum- 
Based Vocational 
Assessment Process 



6 



7 



175 BEST COPY AVA. ABLE 



Assessment of Students with Handicaps in 
Vocational Education: A Curriculum-Based Approach 



Module 5 



MODULES ; 

_ Monitoring a [ 
Student's 
Progress in 
the 

Vocational 
Education 
Program j 




Monitoring Student Progress 



Module #5 

Assessment of Students with Handicaps in Vocational 
Education: A Curriculum-Based Approach 1 ' 2 



Prepared by 

Leonard Albright 

California State University, Long Beach 

and 

R. Brian Cobb 

Colorado State University 



This module and the companion training modules are published by the 
American Voca+ional Association, 1410 King Street, Alexandria, VA 22314. 

The materials developed for this training system were made possible through 
a special projects grant from the Office of Special Education and Rehabilita- 
tive Services, United States Department of Education (GO08630O57). However, 
no endorsement by the U.S. Department of Education should be inferred, 

Copyright © 1988 LEONARD ALBRIGHT AND R. BRIAN COBB, Garden Grove, 
California. All rights reserved. No part of this publication may be stored In a 
retrieval system, transmitted, or reproduced in any way, including but not 
limited to photocopy, photograph, magnetic or other record, without the prior 
agreement and written permission of the authors. 



17,' 



Table of Contents 
Module #5 

Page 



Goal and Competencies 1 

Introduction 2 

Purpose 2 

Strategies 2 

Specify Student Assessment/Monitoring Areas 3 

Determine Assessment Procedures 3 

Direct Observation Methods 4 

Table 1: Task Performance Checklist 5 

Table 2: Performance Appraisal for Computer Usage 6 

Knowledge Testing 7 

Table 3: Identifying Sequential Procedures 8 

Frequency of Assessments 9 

Figure 1: Student Progress Chart 10 

Personnel Responsibilities 11 

Table 4: Sample Service Delivery Report 12 

Summary 13 

References 15 




Module 5 
Monitoring Student Progress 



GOAL: The information in this module will assist you 

in establishing a monitoring system which 
produces information about student progress in 
a vocational education program and the ade- 
quacy of special services. 



COMPETENCIES: As a result of completing this module, you and 

your CBVA team members will: 

1. Identify the types of student progress in- 
formation that need to be collected; 

2. Develop appropriate assessment instru- 
ments and procedures for collecting stu- 
dent progress information; 

3. Determine the frequency of information 
collection activities; 

4. Identify the various personnel to be in- 
volved in collecting and reporting student 
progress information; 

5. Report student progress information in a 
variety of formats; and 

6. Utilize student progress data to make 
decisions about changes in instructional 
programming. 
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Introduction 

After identifying the special services needed by the student (Module 4), 
procedures need to be established for monitoring the progress of the stu- 
dent as she/he participates in a vocational education program. Information 
collected from these procedures is necessary for determining if existing ser- 
vices are adequate or whether modifications are needed to the curriculum 
and/or the type of special services being provided. Therefore, monitoring 
procedures need to be tied directly to the student's special services plan 
(Vocational IEP) and should be conducted on a regularly scheduled basis. 

When initially examining this phase of assessment, a common reaction 
by both the vocational instructor and the person in charge of the student's 
special services plan is that assessments of student progress are already 
occurring Upon closer inspection, however, what is usually found is that, 
yes, each party has his/her own procedures, but they are being done in a 
separate and informal manner. What is missing is a coordinated assess- 
ment and monitoring process between both parties; one that is based on 
mutual agreement of specific areas in which student progress data will be 
collected and reviewed on a regular schedule. 

Purpose 

The purpose of this module is to present a series of guidelines and con- 
siderations in designing a coordinated process for monitoring student 
progress in vocational education and training programs. Information rela- 
tive to what should be assessed, when , how, and by whom is the focal point 
of this module. 

Strategies 

As vocational education and special services personnel work on estab- 
lishing a coordinated student monitoring process, several basic considera- 
tions need to be addressed. The first consideration is a determination of the 
specific areas in which student progress will be monitored. Once these have 
been determined, attention needs to be given to the procedures to be used 
in conducting assessments of student progress, the frequency of assessment 
activity, and the personnel responsible for completing the assessments and 

'Special appreciation ic extended to Lynn Safarik, C3VA Project Coordinator at 
California State University-Long Beach, for preparing the field-teat ver- 
sion of this module. J / j 
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reviewing the results. A more detailed description of each of these con- 
siderations follows. 

Specify Student Assessment/ MonitorL ^ Areas 

Through earlier assessments (Module 4), the types of special services 
to be provided to the student were identified. Now, however, further 
specification of these services may be necessary to determine what needs 
to be assessed and where these assessments will occur. If, for example, it 
has been determined that a student needs special help in improving work 
habits, then a breakdown of observable behaviors which demonstrate 
desirable work habi's in the vocational program could be identified. These 
could include such student observables as: 

1. Time on task/off task; 

2. Orderliness of work area; 

3. Organization of materials necessary for beginning a 
task; and 

4. Skills in working independently, without direct instruc- 
tor supervision. 

Or, if a student is to receive special help with reading and/or math in 
the vocational content area, in what specific content areas will progress be 
monitored? And, in which settings will these assessments occur? In this ex- 
ample, such assessments are likely to be needed in the vocational classroom 
and in the special needs services facility. 

Determine Assessm ent Procedures 

In keeping with a basic premise of C VA, every attempt should be made 
to link student assessments directly to the curriculum. Given the perfor- 
mance oriented nature of vocational courses, most assessments of student 
progress can be done through direct observation. The examples just given 
for targeting improvement of student work habits suggest the importance 
of direct observation as an assessment procedure. Yet, less direct measures, 
such as assessing student knowledge of safety practices or tool and equip- 
ment usage via written tests, are also present in vocational classes. Hence, 
assessments of both student performance and student knowledge are neces- 
sary and must be done in a systematic manner, 

Tips for conducting performance and knowledge assessments are sub- 
sequently provided. However, before introducing these, certain criteria 

1 S .i 
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should be applied to the selection and development of assessments for 
monitoring student achievement. These criteria are (Deno, 1985): 

1. Simple and efficient : In order to frequently check or 
monitor student progress, assessments must be easy to 
administer in a relatively short period of time. For ex- 
ample, instead of using one lengthy test on tool and 
equipment identification at ihe end of a unit of instruc- 
tion on this topic, a series of short, spot-check exams 
could be used as instruction is occurring. 

2. Easily understood : Assessment procedures should be 
easy to follow so that the results are clearly and correct- 
ly communicated to students, teachers and parents. 

3. Valid and accurate : To be a true or valid indicator, an 
assessment must measure what it is intended to 
measure. For instance, if the goal is to assess student 
skills in trouble-shooting an automotive engine 
problem, a performance test in the auto shop is going to 
be i. more accurate and truer measure than a written 
test on this topic. 

4. Inexpensive : Since multiple forms or testing materials 
are required for repeated measurement, the costs of as- 
sessment instruments should be minimal. 

Direct Observation Methods 

The use of direct observation is especially valuable when a student is 
learning a new skill or completing a series of steps to perform a certain 
task. This form of assessment, often termed "process assessment," enables 
the observer to pinpoint specific problems and strengths as the student per- 
forms the skill. Corrective steps can immediately be taken, before inap- 
propriate patterns become established. 

One often used instrument for observing and recording student perfor- 
mance is a task performance checklist. As shown in the example on Table 
1, a detailed analysis of the task was completed and a three-column rating 
scale was used in judging the level of student performance. 

The task performance checklist in Table 1 is designed for use in one ob- 
servation of student performance. While appropriate, a similar checklist 
developed for use in recording multiple performances or trials may be more 
economical. Such a checklist is shown in Table 2. Note that another ad- 
vantage of recording the student in multiple trials is that a clear display of 
student skill development over time is readily available to the student, 
parents and teachers. i . 



Table 1: Task Performance Checklist 

Task: Woodworking skills: Cutting of board with crosscut saw 1 

Task Tas k Performance 

Accurate Independent With Assistance 



x. Placed board correctly in vise 








2. Measured to scale 








3. Drew straight line with square 








4. Held saw correctly 








5. Placed saw correctly 








6. Started kerf 








7. Cut board accurately 








8. Checked straightness of cut 








9. Cleaned up work area 









Developed by James Burrows, vocational agriculture teacher, Monroe Area High School, Monroe, Georgia, May, 1987 as part of the 
CBVA field-test program. 



Table 2; Performance Appraisal for Computer Usage 



Client Name: 



Code: X = »ble 
— = unable 



Performance task Date: 

























• 


1. Identifies computer monitor 
























2. Identifies central processing unit 
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K Pulls flnnnv rliftlt firnrii inrlfpf vintihniii" K^n nlincy 

holds by top near label 


























6. Places DOS disk in drive in proper position 


























7. Closes disk drive door 


























8. Turns on monitor 


























9. Turns on central processing unil 



























*An abridged version of a performance appraisal instrument prepared by Raymond Waters, the Rock Creek Foundation, Silver Spring, 
Maryland. Developed as part ofaCBVA pre-service courss* taught through George Washington University, Fall 1987 and Spring 1988 
by Pam Leconte and Carie Rothenbacher. 
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Rating scales and checklists such as those displayed in Tables 1 and 2 
provide baseline information about student performance. However, by in- 
cluding space on this instrument for observer comments, specific informa- 
tion about the type of errors committed and suggested ways for improving 
performance can be recorded and communicated to the student. 

In summary, the use of checklists and rating scales in observing stu- 
dent performance fulfills two important purposes: 

1. They objectify assessment data, resulting in a sound 
basis for making support service decisions; and 

2. They serve as a means of recording progress data, 
providing a well-documented description of the 
student's progress to be reviewed by the vocational spe- 
cial needs resource teacher, parent, or student. 

Knowledge Testing 

Perhaps one of the most frequently used methods for assessing student 
knowledge in vocational programs is the identification test. Identification 
tests can be used to assess a student's ability to identify objects, correct and 
incorrect procedures, elements of a process, or parts of an instrument, 
machine, or product. Examples of identification tests used in vocational 
programs might include: the identification of all the keys on a typewriter 
(office occupations), the identification of the bones in the body (medical 
assisting), or the identification of tools required for transmission work (auto 
mechanics). 

An identification test is particularly useful in assessing a student's un- 
derstanding of the correct sequence of a set of procedures. A test of this kind 
can be devised easily by listinga series of subtasks within a task, and asking 
the student to number the subtasks in the order in which they should be 
executed. An example of this form of testing is shown in Table 3. Other 
types of paper and pencil knowledge tests, such as spelling, vocabulary and 
numerical operation tests are also used to monitor basic academic skills in 
the vocational context. 

Only a few examples of knowledge assessment are mentioned in this 
section. However, these should provide the vocational instructor and voca- 
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Table 3: Identifying Sequential Procedures 



Example Item 13 



13. Number ihe following processes in order in which ihey occur in ihc growing of corn in a field that had been used for corn the previous year. 

Plant 

Disc 

Apply liquid nitrogen 

Apply starter fertilizer 

Apply herbicide 

Apply corn borer killer 

Harvest 

, Cultivate 2nd and 3rd time 

Harrow 

Plow 

Roto hoe 

Cultivate 1st time 



^Vom Measuring student growth by R.C. Erickson and T.L. Wentling, 1976, Boston: Allyn and Bacon, Inc. 
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tional special needs services personnel with some ideas for monitoring stu- 
dent understanding as she/he progresses through the vocational program. 

Frequency of Assessments 

Frequent assessment and monitoring 01 student progress is a central 
feature of CB VA. Since the primary purpose of these assessments is to make 
decisions about the student's success in Ae vocational program, the 
monitoring of student progress should occur as frequently as daily or week- 

ly. 

A progress chart is often used as a means for tracking and displaying 
student growth over time. A progress chart developed for a student in a 
class on writing business letters and memos is shown in Figure 1. This was 
used in charting the student's performance on class assignments during 
the quarter. The aimline (Germann, 1983) shown horizontally on this chart 
represents the instructor's minimally acceptable level of performance for 
this student. 

A set of procedures for use in conjunction with an aimline progress 
chart, like the one displayed in Figure 1, has been adapted from the work 
of Germann (1983). These adapted procedures are as follow d: 

1. Each day or week measure the student's progress. 

2. Keep a record of the measured progress on a tally sheet, 
checklist, or progress chart. 

3. Use the progress record to plot progress points on a 
graph designed to be used for the entire monitoring 
period. 

4. View the student's graph daily or weekly. 

A. Note if the student's progress is above or below the aim- 
line. 

B. If the student's progress is below the aimline, it may be 
necessary to make an intervention. 

5. When the student's progress is below the aimline, a 
decision must be made about whether or not an inter- 
vention is appropriate, and if so, about the nature of that 
intervention. 

A. When using daily measurement, interventions should 
occur when data are below aimline for three to five con- 
secutive days. 

vj'j 



Figure 1: Student Progress Chart 



Per- 
cent 
Grade 

100 

97 

93 

90 

87 

83 

80 

AIM 77 

73 

70 

67 

63 

60 

57 




#1 



#2 



#3 



#4 



#5 #6 

Assignments 



#7 



#8 



#9 



#10 



lDeveloped by Douglas J. Cornfield during the field-testing of Uiis CBVA training program in a vocational special needs pre-service course 
taught in Fall, 1987 at the University of Georgia by Michelle Sarkees ad Nick Elksnin. 
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B. When using weekly measurement, interventions should 
occur when data are below aimline on two consecutive 
evaluations. 

C. These guidelines may he altered as instructor sees fit; 
many times a lack of progress may be due to insig- 
nificant factors, such as vacations or illness, etc. 

6. The fact that an intervention has been made should be 
indicated on the graph. 

7. The intervention should be a substantial change (one 
that has a high probability of causing a student to im- 
prove in the skill area). 

8. When progress is consistently below what is expected, 
frequent changes in strategy should be tried. However, 
no more than one change should be made at once. 

9. The vocational instructor and the special needs resource 
teacher should be reviewing the progress chart and dis- 
cussing possible intervention strategies to improve stu- 
dent performance. 

The preceding progress charting activity is basically the responsibility 
of the vocational instructor or a trained aide, in consultation with the spe- 
cial needs resource teacher. For additional ideas and examples of student 
progress reporting formats in vocational education, see Module 10 in 
Sarkees and Scott (1985). 

In order to maintain an ongoing account of the special services being 
provided to the student, special needs personnel periodically check with 
vocational instructors to obtain this information. A sample "quick-check" 
form similar to the one displayed in Table 4 can be used for this purpose. 

Personnel Responsibiliti es 

Primarily, the responsibility for monitoring student progress is to be 
shared by the vocational instructor and the vocational special education 
rebource teacher. However, whenever possible, the assistance of trained 
aides is desirable, especially when progress must be measured on a daily 
basis or when the number of students being monitored is substantial. The 
possibilities for student self-monitoring or peer monitoring should also be 
considered. In any of these situations, though, training must be provided 
to ensure that proper assessment and monitonng procedures are being fol- 
lowed. 

I9.i 



Student: JancHcins Date: 10/3/88 



Table 4: Sample Service Delivery Report 1 

Contact Person(s): Mary Smith Vocational Program: Auto Mechanics 



The following services were specified in the special services plan for the student identified above. Please indicate the status of these services by placing a check 
in the appropriate categories. 



Type of Special Services 


In Process 


Completed 


No Action 


Comments 


I. Scheduling Modifications 

1. Tutoring in reading related to 
vocational auto mechanics. 

II. Curriculum Modifications 

1. Special math workbook prepared for 

student by special and vocaJonal 
personnel. 

2. Student is completing math 
workbook. 


X 


X 


X 


Teacher aide is helping 
Jane with the course 

tuvt ann ciinnloiriarif dI 

text unu ouppivTuenttn 
materials on an average 
of 4 periods per week. 

Is being typed. 


III. Equipment/Facilities Modifications 

1. Orientation training to alert student 
to presence of hydraulic lifts and 
other potentially hazardous areas. 

2. Installation of fai ets with dials and 
foot controls. 




X 
X 






Special services provided tut not listed in student's plan: None at present 
Do you see a need for addi ional services for this student? No 

If so, please specify these services and indicate if a conference is needed to arrange these services. 
Please return to the special services coordinator by 10/10/88 Thank you! 



Adapted from Albright, L. (1986). Vocational education for the disadvantaged and handicapped. A guide to program administration. 
Arlington, VA: American Vocational Association. 
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Student self-assessment is something that takes place in most vocation- 
al programs, mainly on an informal basis. However, by providing the stu- 
dent with a rating form, like the ones shown in Tables 1 and 2, she/he is 
given a good source for knowing exactly what is expected in terms of per- 
formance. Self-monitoring may be used as a motivational tool in this way. 

In addition to saving instructor time, peer rating can have a number of 
beneficial effects. It has been found that involving students in monitoring 
procedures serves toenlighten them as to the nature of performance evalua- 
tion, and reinforces their own retention of the skill (Erickson & Wentling, 
1976). 

Summary 

This module has presented a variety of guidelines and strategies for 
monitoring student progress in vocational education programs. Several im- 
portant considerations, such as areas to assess and personnel respon- 
sibilities for monitoring student achievement were discussed, as well as 
example instrumentation. However, central to the monitoring process are 
three key principles: 

1. Assessment and monitoring procedures must be coor- 
dinated between vocational and special services person- 
nel; 

2. These procedures must be tied into the student's voca- 
tional curriculum; and 

3. The procedures employed must be efficient, effective 
and conducted in a systematic manner. 

Student progress data collected during participation in the vocational 
program are used for making decisions about instruction and special ser- 
vices. However, information gathered about student performance over the 
course of the program should also be helpful in making decisions about the 
needs of the student as he/she transitions from the vocational program into 
other employment and/or training settings (Module 6). 
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GO AL: Upon completion of this module, you will gain 

an understanding of the knowledge and skills 
necessary to determine the transitional service 
needs of individual vocational education stu- 
dents, and the overall effectiveness of the tran- 
sition services provided to those students. 

COMPETENCIES: In order to cor duct transitional services assess- 

ment and planning, you should demonstrate: 

1. Knowledge of the transitional services 
mandates in federal vocational and special 
education legislation; 

2. Understanding of how to assess the nature 
and intensity of individual student transi- 
tional service needs; 

3. Understanding of the role of transitions 
assessment in the development of an in- 
dividualized transition plan; 

4. Knowledge of the different transitional 
service paths that students may follow in 
moving from school to work or further 
education; and 

5. Understanding of the specific questions 
that circumscribe transitional assessment 
activity. 
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Introduction 

Responding to the transitional services needs of students with hand- 
icaps who are nearing completion of their vocational education program is 
a relatively new consideration in high school service delivery. This priority 
evolved from a number of follow-up studies (c.f. Hasazi, Gordon, & Roe, 
1985; Mithaug, Horiuchi, & Fanning, 1985; Wehman, Kregal & Seyfarth,' 
1985) indicating that, as a general rule, students with handicaps face a very 
uncertain employment future upon exit from high school, much more so 
than their non-handicapped counterparts. Thus, a movement has gained 
momentum since the mid-1980s to place at least some of the responsibility 
on the public schools to help assure that students with handicaps move into 
postsecondary education, training, or employment as smoothly and ex- 
peditiously as possible. 

Indeed, both special and vocational education legislation passed in the 
1980s (P.L. 98-199 and P.L. 98-524 respectively) contained language ad- 
dressing transitional service needs, and specifying a range of potential op- 
tions for responding to those needs. In particular, Sec 101 204(c) (4) of the 
Carl Perkins Vocational Education Act states that every student with a 
handicap who t trolls in a vocational education program must receive: 

". . .Counseling services designed to facilitate the transition 
from school to post-school employment and career opportunities." 

Similarly, the transition initiatives contained in special education legis- 
late ( n .L. 98-199) have contributed to a substantial reshaping of the 
secondary special education curriculum around the country to include such 
services as job development, placement, training, and follow-up services for 
students exiting the public schools (Cobb, Hasazi, Collins, & Salembier, 
1988; Renzaglia, 1986). 

When organizing transitional services for students with handicaps who 
are graduating from vocational education programs, several important 
questions or considerations must be addressed: 

1. What is the range of transitional services that will be 
routinely available for all students with handicaps as 
they near exit from school? 

2. Which school or community agency (special or vocation- 
al education, vocational rehabilitation or developmen- 
tal disabilities) will take responsibility for providing 
these transitional services? 

2>\i 
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3. How will the provision of these services be coordinated 
across these agencies? 

4. Depending on a student's unique needs and handicap- 
ping condition, how will the nature and intensity of 
these transitional services be determined? 

Answers to the first three questions largely reflect local policy decisions 
regarding service delivery specification and coordination. The answer to 
the fourth, however, requires an assessment process. Once again, cur- 
riculum based processes appear to be most responsive to determining the 
nature and intensity of individual transitional services. The "curriculum" 
in this case is the community context in which the student will be function- 
ing, either postsecondary education or training, or competitive, transition- 
al, or supported employment. 

This module contains three parts. The first part will begin with a dis- 
cussion of some of the definitions and considerations associated with com- 
munity placements in either education, training, or work. The second part 
will review a transitional services assessment model (Cobb & Danehey, 
1986), which is consistent with the curriculum based measurement as- 
sumptions outlined in Module 2 of this series of training modules, and with 
the assessment concepts and purposes contained in Modules 3, 4, and 5. 
Finally, a case study will be provided to clarify transitional assessment ac- 
tivity in the overall service delivery structure. 

Definitions and Considerations 

For vocational education students with handicaps who are exiting from 
the schools, the outcomes upon graduation would appear to be that they 
should be in further education, employment training, work, or some com- 
bination of these three. Many students are likely to want to go on to post- 
secondary education programs such as in vocational-technical institutes, 
community or junior colleges, private proprietary schools, or universities. 
Similarly, other students may wish to be enrolled in vocational training 
programs, such as appreuticeships, JTPA training programs, or corporate 
training programs. Finally, other students will want to be employed, either 
full- or part-time, in some form of competitive, transitional, or supported 
work option. Regardless of the type and severity of a student's handicap, 
the full range of these post-school options ought to be made available. 
Several states, for example, have model programs at two-year and four-year 
colleges in which students with severe physical and mental handicaps 
receive academic, social skills, and employment training within the college 
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environment. The point here is that constraints on the avaiiamm.y 01 poat- 
school options for the full range of exiting students with handicaps are 
defined not by lack of student abilities, but by local policy, fiscal pressures, 
and underdeveloped services coordination across agencies. 

Gains have been made in recent years, however, especially in the dif- 
ficult area of services coordination. The transitional service delivery ksues 
associated with student movement from school to post-school environments 
have been articulated by Will (1984), and can be broadly categorized into 
three transitional paths. Figure 1 displays those three paths and highlights 
the service delivery and assessment issues in terms of the need for, and in- 
tensity of services after a post-school option for a student has been iden- 
tified and selected. 

The first path, "no special services," would typically apply to students 
with mild handicaps who participated in vocational education programs 
and who leave the school system with competitive employment, or further 
education or training firmly in their grasp. These students would require 
no special services from either special or vocational education, or adult ser- 
vices agencies such as vocational rehabilitation. However, they may still 
utilize generic community services such as Job Service, or other community 
agencies in whatever endeavor they may have chosen. 

The second path, "time-limited services," would involve the planning at 
the secondary level for the provision of short-term assistance at the adult 
services level for a wide range of vocational education graduates with mild 
to moderate disabilities. Again, these students would find themselves in 
post-secondary education or training, or in transitional employment. Tran- 
sitional employment is placement on a job with whatever direct support 
which might be necessary to ensure success in that position. This support 
is anticipated to be short-term, however, usually for several weeks to 
several months in duration. Adult services agencies such as vocational 
rehabilitation, community mental health agencies, and JTPA programs 
provide these kinds of services. 

Finally, "ongoing services" would potentially be appropriate for those 
students graduating from vocational education programs who were more 
moderately to severely handicapped. Transitional services for these in- 
dividuals would again share a common goal of post-secondary education or 
training, or supported employment. Supported employment involves the 
provision of the same types of services as in transitional employment, ex- 
cept that they are envisioned to last indefinitely, or for as long as they are 
needed by the individual. This "open-ended" plan for service delivery 
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Figure 1 : Transitional Pathways 

School Ayed 
Disabled Population 

Less Severity of More 

« » 

Severe Disability Severe 



Transition 




Time limited 
services needed 



Ongoing lifetime 
services needed 



i 



Competitive Transitional 
Employment Employment 

OR 

Post -Secondary Education or Training 



Supported 
Employment 



Transition 



Adapted from Will, M. (1984). Bridges from school to working life: Programs for the Handicapped. 
Washington, D.C.; U.S. Department of Education, Office of Information and Resources for the 
Handicapped. 
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precludes the use of some agencies such as vocational rehabilitation or 
JTPA programs, since these agencies can only take individuals into their 
programs for fixed amounts of time. However, a number of supported 
employment agencies, such as community mental health/mental retarda- 
tion programs, developmental disabilities programs, or Association for 
Retarded Citizens programs provide such services, frequently in collabora- 
tion with vocational rehabilitation agencies. 

Transitional Assessment and Planning Proces s 

Clearly, decisions about the selection of a particular transitional path 
for each student, the nature of the transitional services to be provided, and 
who will be responsible for providing those services require an individual- 
ized process involving assessment and cooperative planning. The 
remainder of this module will highlight a process for transitional assess- 
ment and planning. This process could be cooperatively implemented be- 
tween vocational and special education at the school level, and adult 
services agencies at the community level, to adequately address the tran- 
sitional service needs of students who are leaving the public schools. The 
transitional services assessment and planning process can be broken into 
placement/planning, and monitoring/evaluation functions. 

Pla cement/Planning 

Transitional assessment for the purpose of placement/planning addres- 
ses three major considerations: 

1. In which outcome(s) [either postsecondary education, 
training, or employment] will the student be function- 
ing upon graduation? 

2. Which transitional path reflects the level of support the 
student will likely require tc tmccessfully integrate into 
that outcome? 

3. What services will the student require at his/her 
sel acted option? 

Regarding the first major consideration, several assessment questions 
should guide the process: 

1. What are the student's aspirations? How realistic are 
they? 

2. What are the parents' major concerns and future expec- 
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tations for their son or daughter? 

3. What are the available options from which the student 
could choose? 

4. To what extent is the student likely to be engaged in a 
combination of options? 

To determine student aspirations, parent expectations, and the 
likelihood for a combination of education, training, and work, the assess- 
ment process would involve ongoing dialogue between special and voca- 
tional education personnel on the one hand, and the student and his/her 
parents or guardians on the other. One important activity involves a review 
of cumulative data that have been maintained on the student through 
his/her vocational education experiences. These data are often reported in 
the form of student competency profiles. For examples of these cumulative 
reporting forms, see Module 10 in Sarkees and Scott (1985). 

Dialogue among special and vocational personnel, student and parents, 
could and should take place in the development and monitoring of the In- 
dividualized Transitional Plan (ITP). The ITP has been adopted in many 
states as an integral component of the Individualized Educational Program 
(IEP) and provides an opportunity for educational personnel and families 
to jointly plan for the future beyond school. To find out the ITP format for 
your district or state, you should contact the secondary consultant in your 
district's special education department. 

Similarly, vocational education personnel can help to focus a student's 
aspirations and parental expectations through their participation in the 
ITP process, as well as through individualized career guidance and coun- 
seling provided as their response to the mandate in the Perkins Act. While 
a number of standardized instruments for determining career interests 
have been developed specifically for students with handicaps [e.g., The 
Reading-Free Vocational Interest Inventory; The Vocational Interest and 
Sophistication Assessment; The Self-Directed Search (revised version)], it 
is likely that the most useful information about post-school interests and 
service needs will be derived from interviews, from cumulative cVa that 
have been maintained on the student during his/her vocational education 
program (Sarkees & Scott, 1985), and from direct observations of the stu- 
dent in community-based, work experience settings. 

To determine the options available for a student who is finishing school 
would again require a thorough understanding of the student and his/her 
strengths and limitations, as well as a knowledge of the adult service agen- 
cies providing support, and the job opportunities in the community. Clear- 
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ly , special educall > personnel would take the primary role in determining 
the types of agencies which could provide support to the student, and in 
connecting the student to those agencies. Vocational education support per- 
sonnel could play an important role in helping the student understand the 
job opportunities in the community, as well as connecting the student with 
potential employers in a manner similar to the informed connection 
provided through cooperative vocational education experiences. 

To address the second major consideration, which is determining the 
appropriate path a student would follow in the transitional process, several 
different assessment questions must be addressed: 

1. How well does the student's employment or education- 
al history while in school assist in identifying the ser- 
vices will be required in his/her community placement? 
For example, if the student worked part-time while in 
school, it is likely that his/her employment record can 
provide important information about how much support 
may be required to support his/her full time job. 

2. How well will any identified service needs be provided 
generically in the community, or by the family/friend 
network? 

Again, the ITP process should provide the mechanism by which educa- 
tional personnel and families can determine the extent of support a stu- 
dent might require in whatever endeavor he/she may enter into upon 
leaving school. No standardized instrument can determine this level of sup- 
port, since it will vary for an individual student depending on the choice 
he/she selects. For example, a student may be capable of performing the 
duties of one particular job with very little follow-up support, but may need 
much more support for a job with greater demands and more difficult work 
routines. Ultimately, the intensity of support a student will need in a post- 
school placement can be validly assessed only through a situational assess- 
ment process — namely, putting the student in that environment and 
determining support levels through direct observation of performance 
(Gaylord-Ross, 1986). 

It should be evident that determining the exact nature of the services 
to be provided, the third major consideration outlined on page 6, must also 
be the product of a situational assessment. 

The specific assessment questions to be addressed here are: 

1. What agencies exist that can provide an entire range of 
transitional services? 
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2. Which of these agencies can deliver the specific services 
that this individual student requires, and for the dura- 
tion of time initially projected? 

3. What types of community associations exist which could 
be enlisted for aid if advocacy is needed? 

Determining the nature of services to be provided requires a thorough 
understanding of the agencies in the community which can provide those 
services, and fixing responsibility for delivering those services prior to the 
student leaving the school environment. By and large, special education 
personnel should take the lead in determining this fixed point of respon- 
sibility, although in some employment and vocational training situations, 
vocational education personnel can provide valuable assistance. 

To conclude, the most difficult issue with which school personnel must 
grapple is balancing the potentially legitimate service needs of an in- 
dividual beyond what is generically available in the community with the 
equally likely alternative to "over- program" specialized services for the stu- 
dent. Young adults leaving the public schools must be given the opportunity 
to assume greater responsibility and risks for their lives. It seems likely 
that the single most important function of school personnel in transitional 
assessment and planning would be their informed, sensitive and profes- 
sional judgment at this point. 

Monitoring/Evaluation 

Special education and vocational education personnel need to engage 
in monitoring and evaluation procedures to assess the impact of their tran- 
sitional services. Specifically, the judgments made about the nature and 
intensity of services required by students with handicaps need to be 
reviewed for their appropriateness. It is unreasonable to expect school per- 
sonnel to take responsibility for monitoring the quality of individual ser- 
vices provided by adult services providers. However, school personnel 
should have some notions of the overall effectiveness of different agencies 
at the adult level, so that they can make judgments and recommendations 
to future graduating students and their parents about which agencies can 
be counted on to provide high-quality services. 

The primary method(s) which can be used to assess effective school- 
based and community-based transitional services would be follow-up inter- 
views, both formal and informal, of former students, employers, and adult 
services providers. Typically, questions on such a follow-up interview could 
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focus on satisfaction with school, use of adult services, status of employ- 
ment, utilization of further vocational training programs, future job goals, 
and general satisfaction with community living arrangements. The 
framework in Module 7 for evaluating the overall CB VA system should also 
provide important information about the quality of transitional services. 

A Case Illustration 

The following hypothetical case example may help to clarify how tran- 
sitional service assessment and planning might be delivered: 

David is a 19 year-old, mildly mentally retarded student en- 
rolled in his final year of a Building Trades Program at a 
secondary, regional vocational center. His program was 
modified significantly to include intensive training in the 
areas of roofing, sheathing application, and flooring instal- 
lation, along with general framing content. During the Fall 
semester, the vocational special needs teacher (VSNT) 
responsible for transition worked with the cooperative 
education coordinator at the center and identified a local con- 
tractor who was willing to hire David, given assurances that 
he could perform the work. David and his paients were in- 
terviewed and agreed that he would like to work as a car- 
penter. David's vocational teacher indicated that he believed 
David could perform well at the job, given initial support to 
make the transition from a building trades classroom to an 
actual job site. 

A transition plan had been developed for David during his 
sophomore year in school. In it, goals and objectives relative 
to work, community -living, social skills, and leisure and 
recreational activities were targeted. The work component 
of the transition plan had focused on construction work as a 
priority training area for David, and he was placed in the 
building trades program in response to that interest. The 
transition plan also included a recommendation that David 
be placed in, and be receiving support at a full-time, per- 
manent job upon his graduation from high school. Now it was 
time to implement the final assessment and planning com- 
ponent of the transition plan. 

To begin this final component, David was placed in a 
modified cooperative work education (CWE) program at his 
future job site during the Spring semester. He initially at- 
tended the building trades class in the morning, and was at 
the job site during the afternoon. As the semester progressed, 
David spent increasing amounts of time at the job site, such 
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that by the end of the semester, he was working there full- 
time. 

The arrangement made with the employer was especially 
conducive to gradually integrating David into his work 
routine. First, the VSNT negotiated the initial work routine 
with the employer, and learned the tasks over the span of 
sever?! Jays. David was then brought to the work site, 3hown 
the vork routine and asked to perform each of the tasks. The 
VSIsT observed which tasks David could do, and those which 
required further training. He then developed an instruction- 
al program, focusing on those skills that needed further 
development, using a behavior analytic approach to the 
training. Data were kept each day on David's improvement 
with each of the tasks, and all tasks in the work routine were 
performed to the employer's expectations, either by David 
alone, or by a combination of efforts by both David and the 
VSNT. David received his negotiated, entry-level salary for 
the hours he worked, and the employer had his work com- 
pleted. 

By the end of the school year, David was nearly able to work 
independently and at normative standards. At that time, a 
job coach was contracted through the resources of the local 
vocational rehabilitation office. The job coach continued to 
work with David, gradually fading his presence through the 
summer. Then, a follow-up, follow-along schedule was 
negotiated with the employer which continued to assure a 
supportive environment until David was totally inde- 
pendent. 

The case study describes a service delivery process, but it also describes 
how curriculum-based vocational assessment functions within that 
process. Assessment for placement in the construction job was made 
through interviewing and reviews of prior performance. Assessment for 
planning transitional services occurred by observing David's performance 
on the job and identifying the discrepancy between industrial standards 
and that performance. Monitoring and evaluating David's progress was ac- 
complished through direct observation of his work and through informal 
interviews with David and his employer to determine each individual's 
satisfaction with the placement. Module 7, which follows this module, 
provides additional information about how the evaluation process can 
work. 
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Module 7 

Evaluating the CBVA Process 

GOAL: The information and guidelines in this module 

will assist you and district team members in 
evaluating the efficiency ***d effectiveness of 
the CBVA process in pi^viding appropriate 
vocational education and support services to 
handicapped students. 

COMPETENCIES: As a result of completing this module, you and 

your team members will: 

1. Identify the central evaluation questions 
for each component or stage of the CBVA 
process; 

2. Determine the sources for obtaining infor- 
mation to address the evaluation ques- 
tions; 

3. Specify the methods and timelines to be 
used in collecting and reviewing evalua- 
tive information; 

4. Involve key personnel in the evaluation ac- 
tivities; and 

5. Prepare an improvement plan for the 
CBVA process based on the results of the 
evaluation. 
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Introduction 

The information and activities provided in Modules 1-6 were designed 
to aid local school personnel in establishing a curriculum-based process for 
assessing student needs in vocational education programs. Module 1 
provided a set of steps to follow in establishing a CB VA process in your dis- 
trict or agency. Basic information about CBVA and this training program 
was offered in Module 2. 

Important guidelines and strategies for implementing CBVA proce- 
dures were described in Modules 3-6. More specifically, Modules 3 and 4 
focused on assessment practices used for student placement and program 
planning purposes. While this initial level or stage of assessment is criti- 
cal, procedures for monitoring student progress in a vocational setting must 
also be in place to insure that appropriate services are being provided to 
the student. Such procedures were contained in Module 5. Equally impor- 
tant are assessment activities which help in determining the services 
needed by the student to make a successful transition from the vocational 
education setting to the workplace and/or an advanced training program. 
Transitional asL^ssment functions were the focus of Module 6. Collective- 
ly, then, the information presented thus far in the module series establishes 
a CBVA process for students at three important points: 

1. Prior to entry into specific vocational programs; 

2. Daring their participation in vocational instruction; 
and 

3. Upon exiting from the program to work and/or further 
training. 

Purpose 

The intent of this module is to provide a set of procedures for evaluat- 
ing the CBVA process that you have been developing through this training 
program. In contrast to the preceding modules, this module requires you 
to shift gears; from a focus on organizing specific assessment activities 
within the CBVA process to a retrospective examination of how the process 
is now organized and functioning in your school or agency. Such examina- 
tion should be helpful in determining how extensive the local assessment 
process is and in pinpointing particular strengths and weaknesses of the 
process. One important outcome of this activity is to design a plan for im- 
proving the existing CBVA process based upon the findings from the 
evaluation. 
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Pr ocedures 

The evaluation of a newly installed CBVA process essentially involves 
a series of steps or procedures which examines how efficiently and effec- 
tively the assessment process is being implemented in an educational set- 
ting. The basic question addressed through this type of progress evaluation 
is "How are we doing in implementing this program?" (Albright, 1986; Wen- 
tling, 1980). The steps that follow provide a pathway for answering this 
question, as well as other more specific co. jerns related to the CBVA 
process. 

Step h Organize a CBVA Evaluation Team 

The formation of a team to evaluate the local CBVA process is recom- 
mended. Several reasons for mounting a team effort include: 

1. Many different audiences are involved in and/or af- 
fected by a CBVA process, such as counselors, special 
needs resource personnel, vocational teachers, ad- 
ministrators, students and parents. The input of key 
representatives from these audiences o groups is neces- 
sary for obtaining a balanced picture of the CBVA 
processes in a local setting. 

2. The team approach capitalizes on the special skills of 
each member. Guba and Lincoln (1981) note that the 
sharing of responsibilities for the evaluation among 
team members enables the special talents and interests 
of each member to come forth. For example, whereas one 
member may be especially skilled in interviewing, other 
members may be particularly skilled in planning the 
overall evaluation and/or in analyzing and interpreting 
information. This collective venture tends to avoid the 
pitfalls of expecting one person to be skilled in all phases 
of an evaluation effort. 

3 . A team approach permits the use of multiple data or in- 
formation collection strategies within a reasonable time 
frame. That is, by dividing up the evaluation respon- 
sibilities, the workload is shared and more work is like- 
ly to be accomplished in a shorter period of time than if 
one person was given the sole responsibility for doing 
the evaluation. 
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4. Lastly, the involvement of multiple audiences in the 
evaluation effort is likely to enhance its credibility and 
use (Joint Committee on Standards for Educational 
Evaluation, 1981). Since a CBVA process requires coor- 
dinated service delivery among personnel from various 
audiences, these audiences need to be considered in all 
phases of the evaluation; from initial planning of the 
evaluation through the development and implementa- 
tion of a program improvement plan. 

Team Size and Composition 

To insure maximum productivity, the team should consist of three (3) 
to six (6) members. In addition to the person who has the designated respon- 
sibility for managing the local CBVA process, representatives from at least 
two of the groups listed below should be on the team: 

1 . Vocational education faculty; 

2. Special education personnel responsible for manage- 
ment of IEPs for vocational students; 

3. Vocational counselors; 

4. Students; 

5. Parents; and 

6. Adult service providers (e.g., rehabilitation services per- 
sonnel). 

The selection of key personnel to serve as members should, of course, 
be determined within the local program setting. However, when selecting 
team members, three important considerations are: 

1. Their level of involvement in and commitment to the 
CBVA process; 

2. Their interest and skills in planning, conducting and 
using the results of the evaluation; and 

3. Their relative position (formal and informal) in effect- 
ing change within the local district. 

Finally, as to whom should serve as the team leader, the person respon- 
sible for managing the local CBVA process is the most logical choice. If, 
however, there exists a serious concern about the potential of biasing the 
evaluation by having this person head the team, then the selection of 
another team member may be desirable. 
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Step 2 : Focus the Evaluation 

Once team membership is finalized, then the team meets to specify or 
"focus" the evaluation. This focusing activity involves a series of decisions 
about what needs to be evaluated, how and when the evaluation will be con- 
ducted and who will be involved in this effort (Brinkerhoff. Brethower, 
Hluchyj & Nowakowski, 1983). 

Team Orientation Activity 

Before the team can begin to focus the evaluation, the members must 
first have a common understanding of how the CBVA process is organized 
within the district. To achieve this mutual understanding, the manager of 
the local CBVA process should present an overview of how CBVA is present- 
ly organized. This presentation would include information about the 
various purposes of assessment, when and what types of assessment ac- 
tivities occur, and which personnel conduct these assessments. The CBVA 
Implementation Guide shown in Table 3 of Module 2 could be adapted for 
this presentation. 

Establish Key Questions and Concerns 

The focusing decisions occur following a presentation and team discus- 
sion of the local CBVA process. While the overall purpose of the evaluation 
is to determine how well the local process is operating, specific questions 
and areas of concern within each of the major stages of a CBVA process 
should be considered as part of the evaluation plan. Example Document #1 
provides a series of sample content items to examine in a review of each 
CBVA stage, as well as a format team members could use in summarizing 
their findings. Additional content items for each CBVA stage could be 
generated by reviewing the information offered in Modules 1-6. 

Identify Sources of Information 

After establishing the key questions and concerns for the evaluation, 
decisions need to be made about how and where relevant information can 
be obtained to answer these key items. Table 1 identifies four major sour- 
ces of information and some techniques available for collecting this infor- 
mation. These general sources should be adapted by the team in specifying 
their information sources and techniques. Additional tips about selecting 



Table 1: Major Sources of Information 



1 



Source 



1. People 



Techniques 



Types to Consider 



• Conversation (face-to-face, telephone) 

• Structured interviews 

• Questionnaires 



* Those who are held accountable for out- 
comes (e.g., head of assessment services, 
administrator of vocational education) 

, Those directly involved in planning 
and/or delivering assessment services 
(e.g., vocational and special education 
teachers, counselors) 

# Those who are to benefit from services 
(e.g., teachers, students, parents) 



2. Performance data 



• "Eyeballing" student assessment data 

• Observation 

• Interview 

• Content analysis of student assessment 
and progress reports 



m Reports on student placement, progress 
and achievement in vocational education 

# Obs^ vations of performance 



3. Documents 



• Counting numbers of documents (e.g., 
letters sent to students with handicaps 
and parents about options in vocational 
education) 

• Content analysis of student program 
plans and progress reports 

• Assessing quality of printed materials 



# Reports 

, Proposals 

# Student program plans (e.g., IEP and 
ITP documents) 

# Student assessment charts 



4. Context (the setting) 



• Observe 

• Interview key informants 

• Read reports 



# Facilities 

# Schedules 

# Organizational plans 

# Attitudes of personnel 

# Formal and informal power structures 



Adapted from BrinkerhofT, R.O., Brethower, D.M., Hluchyj, T., & Nowakowski, J.R., (1983). Program evaluation: A practitioner's guide for 
trainers and educators. Boston: Kluwer Nijhoff Publishing. 
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Prepare an Evaluation Work Plan 

The last activity in focusing the evaluation is to formulate a work plan 
for the evaluation. This document is to serve as a summary of the impor- 
tant focusing decisions made by the team members. Also included in this 
work plan is a listing of timelines for completing the evaluation and the 
personnel responsible for various activities. Example Docum ent #2 con- 
tains an illustration of an evaluation work plan. 

Step 3; Conduct the Evaluation 

The evaluation work plan should serve as the team's guide for conduct- 
ing the evaluation. Since this work plan lists the activities which need to 
be completed by each team member within a designated time frame, each 
member should have a copy of the work plan. Moreover, the chairperson of 
the committee can use this document to monitor the progress of the evalua- 
tion. 

The remainder of this section provides additional tips to consider in or- 
ganizing and conducting the evaluation. 

Use of Multiple Information Sources 

A recommended evaluation practice is to use more than one source to 
verify pertinent information (Maher & Bennett, 1984). For example, in ex- 
amining the adequacy of established procedures for providing transitional 
services, multiple sources such as conversations with selected vocational 
teachers, adult service providers and students and a document review of a 
sample of transitional service plans may be appropriate. By checking more 
than one source in your review, a more complete and accurate accounting 
of the CBVA operation can be obtained. Such practice is especially impor- 
tant in a review of a CBVA process; a process which is based on coordinated 
service planning and delivery among a host of players (e.g., teachers, secon- 
dary and adult service providers, counselors, parents, students and ad- 
ministrators). 
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Instrument Selection and/or Development 

In order to collect the information necessary, the team will likely need 
to select or develop specific data collection instruments. If, for example, the 
team wants to survey a sample of vocational teachers to determine their 
understanding of and involvement in monitoring student progress (Module 
5), then a survey instrument may need to be developed. In order to avoid 
lengthy and unnecessary developmental work, invest some time in check- 
ing whether similar instruments already exist within the district or are 
available through other sources such as in like programs in other districts 
or agencies, libraries and resource centers. 

If, however, a specific instrument or instruments need to be developed 
by the team, special attention needs to be given to validity and accuracy 
matters. The guidelines shown in Table 2 illustrate the steps commonly 
used in instrument development. 

Coordination of Data-Collection Tasks 

To ensure that the team members collect the necessary information in 
an appropriate manner, a team meeting should be held to clarify this im- 
portant function. The topics to be covered at this meeting are: 

1. Data collection instruments and procedures; 

2. Obtaining access to information sources; and 

3. Timeline for completing data collection activities. 

Discussion about data collection instruments and their use should 
provide team members with an understanding of how to use the instru- 
ments and record pertinent data. 

In particular, however, special attention must be given to "access to 
data" considerations. Making arrangements for interviews with various 
school personnel, access to pertinent documents (e.g., student assessment 
reports, IEPs), and procedures relative to observing assessment services 
and practices are three important pieces of information for the team mem- 
bers. Finally, the timeline for completing data collection activities needs to 
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Table 2. Guidelines for Developing Evaluation Instruments 

Developing Your Own Instrument 



1 . List specifications for the instrument. 

Content What items should it address? 

Precision How precisely and surely must it measure? 

What kinds of decisions will depend on the 
data produced? 

Language Will the instrument be read by 4th graders or 

Requirements Ph.D. candidates? Handicapped persons? 

Bilingual persons? 

Analysis Machine scoring? Special coding? Analysis 

Planned along with other instruments? 

Demographic How much will the instrument need to record 

Data Needs information about respondents themselves or 

the administration setting? 

2. Clarify the conceptual design/basis for the instrument. An instrument shouldn't be a 
haphazard collection of items and directions. It needs a conceptual design—the "glue" 
that hangs it together. Some examples are: 

• "Respondents will be asked to recollect their initial ability levels 
before the workshop, then rate their growth on each of the 
several workshop objectives." 

• "The six typical behavior problems will be presented and respon- 
dents will outline a proposed treatment strategy. Responses will 
be scored according to how well respondents incorporate 
Schwartzian theory." 

• "The questionnaire will list many resources, some of which the 
agency has disseminated. Then, it will ask people to check those 
they use most often." 

3. Block out ('^blueprint") the instrument. Outline the major sections, where 
you'll want directions, how many items you want for each objective, etc. 

4. Produce a draft of the instrument. 

5. Get the draft reviewed. Have it checked for ease of reading and clarity, con- 
tent, technical flaws (e.g., dual stems, overlapping response categories). 

6. Revise the draft (Note: the more often you do this and Step 5, the better it 
will get). 

7. Try out the instrument to be sure it has sufficient reliability and validity for 
your needs. 

8. Revise again, and try-out again until 



Adapted from R.O. Brinkerhoff, D.M. Brethower, T. Hluchyj, & J.R. 
Nowakowslri (1983). Program evaluation: A practitioner's guide for 
trainers and educators (pp. 97-98), Boston: Kluwer Nijhoff Publishing. 

2? 7 



10 



Module 7 

Curriculum-Based Vocational Assessment 



be emphasized, along with the date the team members are to meet to report 
on their individual findings and prepare a composite team report. 

Step 4 : Prepare CBVA Evaluation Report 

A team meeting should be scheduled immediately following the dead- 
line established for individual team members completion of data collection 
and recording activities. The purpose of this team meeting is two-fold: 

1. To discuss their individual findings and observations; 
and 

2. To prepare a composite team report which summarizes 
the major findings and recommends a plan for improv- 
ing the existing CBVA process. 

A sample format for organizing a CBVA evaluation report is shown in 
Example Document #3 . 

Dissemination and Use of Evaluation Results 

If, indeed, key personnel from the various groups associated with CBVA 
are involved in organizing and conducting the evaluation, then they are 
also likely t" be the persons who will make maximum use of the evaluation 
results. However, provisions for communicating the results of the evalua- 
tion *o other important decision-makers should be considered by the team. 
Written and/or oral summary presentations to local board members and to 
the external agency that helps to support local CBVA activities (e.g., state 
division of vocational education) could prove helpful to the total CBVA ef- 
fort in the district. 
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Sample CBVA Evaluation Instrument 



Team member name: 



Instructions: 



Listed below is a series of items to be examined in your review of the CBVA process in our district. Afler collecting the in- 
formation needed for each item, please rate each item according to the following numerical code: 



1. Outstanding: Exceeds stated criteria 

2. Adequate: Meets stated criteria 

3. Needs improvement: Does not meet stated criteria 

4. Not present: No evidence that practice exists 

5. Not applicable: Does not apply 



Assessment stage 



Assessment practice 



Code # 



Comments 



Stage 1: Assessment 
during program place- 
ment and planning 



Communicating program options (Module 1: Part II) 

1. Procedures for communicating vocational education options to 

students with handicaps and their parents are established 
and used. These procedures clearly identify: A. what types of 
information are to be communicated; B. the personnel respon- 
sible for communicating this information; and C. the persons 
who are to recieve Hms information. 

2. Procedures are in place for evaluating the effectiveness of the 

communicating vocational education options program. 

S tudent placement (Module 3) 

3. A vocational assessment team assesses the vocational inter- 

ests and needs of individual students. 

Student program planning (Module 4) 

4. The strategies used for assessing student service needs in voca 

tional education are established and are done in a systematic 
and individualized manner. 

5. The results of these assessments are present in the program 

plans for individual students. 
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Assessment stage 

Stage 2: Assessment 
during program 
participation 



Stage 3: Assessment 
near program com- 
pletion 



Sample CBVA Evaluation Instrument (Continued) 

Assessment practice 



Code # 



Comments 



Monitoring student progress (Module 5) 

6. Student progress in vocational program is recorded and com- 

municated on a regular basis. 

7. Student progress information is being used in making 
decisions about the appropriateness of special services for 
students with handicaps. 

Transitional service assessment and planning (Module 6) 

8. Procedures are in place for determining the transitional ser- 
vice needs of individual students. 

9. Procedures for follow-up of students are in place and com- 
pleted on a regular basis. 
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Summary Observations 

1. Please state what you consider to be the major strengths of the CBVA process. 



2. Please indicate specific areas within the CBVA process that need improvement. 



3. Please list, in order of highest priority, your recommendations for improving CBVA practices. 



Priority Area 



Recommendations 
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Example Document #2 
Evaluation Work Plan 




Evaluation Work Plan 

Team members: (Place asterisk* next to name of chairperson) 



1. 



Name 



4. 



Name 



Position 



Name 



5. 



Position 



Name 



Position 



Name 



6. 



Position 



Name 



Position 



Position 



Evaluation activity 



Personnel involved 



1. Select evaluation 
team members 

2. Conduct team orienta- 
tion and focusing 
meeting 

3. Prepare and dis- 
tribute evaluation 
work plan to team 
members 



Head of CBVA process and coordinator of 
vocational special needs services 

Team chairperson 



• Team chairperson 



Completion date 



11/1/88 



11/15/88 



11/20/88 
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Evaluation activity 



4. Prepare evaluation in- 
struments 



5. Team meets to review 

and finalize collec- 
tion procedures and 
timelines 

6. Individual team mem- 
bers collect evalua- 
tive data 

7. Team meets to review 
individual findings 
and prepare a com- 
posite report 

8. Typed copy of com- 
posite report is sent 
to team members for 
final review and en- 
dorsement 

9. Final report 
presented to school 
board 



Evaluation Work Plan (continued) 

Personnel involved 



Two team members will prepare interview guide 
for use with selected vocational teachers and 
support service personnel 

Chair and one team member will prepare 
evaluation instrument to be used by team 
members in reviewing CBVA process 

Team members 



• Team members 



• Team members 



Chairperson for distributing report; team 
members review and return document to 
chairperson 



• Chairperson presents report to school board 



Completion date 



12/15/88 



12/15/88 



12/20/88 



1/31/89 



2/5/89 



2/10/89 



3/15/89 
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Example Document #3 
CBVA Evaluation Reporting Format 




22 



Modulo 7 

Curriculum-Based Vocational Assessment 



(Report Cover Page) 

Evaluation of the 
Curriculum-Based Assessment Process 

At 

(School/Agency Title) 



Prepared By 



(Name) 



(Name) 



(Title) 



(Title) 



(Name) 



(Title) 



(Name) 



(Title) 



(Name) 



(Title) 
Date-- 



24,1 



Module 7 

Curriculum-Based Vocational Assessment 
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Description of Evaluation (2-3 Pages) 

L Introduction Identify why the evaluation was con- 

ducted, when it was conducted, and the 
sources and procedures used in collecting 
evaluative information. 

II. Summary of Describe m^jor strengths and shortcom- 
Major Findings iags of present CBVA process. 

III. Recommendations State highest priority areas for improve- 

ment and suggest how and when these im- 
provements should be accomplished. 
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